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FOREWORD 


Critical  thinking  skills  are  essential  to  the  effective  performance  of  military  leaders. 

Army  XXI  will  be  faced  with  a  diversity  of  missions,  faster  pace,  greater  uncertainties,  as  well  as 
changing  equipment,  digital  capabilities,  and  organizational  structures.  All  of  these  will  place 
increasing  demands  on  the  planning  and  decision-making  skills  of  Battle  Commanders  and  their 
staff.  Only  through  critical  thinking  can  these  leaders  learn  how  to  adjust  to  the  novel  and 
unpredictable  situations  faced  on  tomorrow’s  battlefields  and  to  the  various  roles  played  by  our 
Army.  Despite  the  vital  nature  of  critical  thinking  skills,  our  knowledge  of  its  fundamental 
aspects  is  somewhat  limited.  Some  fundamental  issues  on  which  there  appears  to  be  little 
consensus  include:  a  definition  of  critical  thinking;  the  extent  to  which  it  is  possible  to  train 
leaders  to  think  critically;  how  to  most  effectively  measure  critical  thinking;  and  how  to  integrate 
critical  thinking  into  current  Army  practices. 

In  an  effort  to  address  some  of  these  issues,  the  U.S.  Army  Research  Institute  hosted  the 
Army  Workshop  on  Critical  Thinking  Skills  for  Battle  Command  at  Fort  Leavenworth,  KS  in 
December  2000.  The  purpose  of  this  workshop  was  to  bring  together  experts  in  critical  thinking 
research  and  battle  command  to  discuss  both  the  latest  research  on  critical  thinking  skills  and 
how  critical  thinking  skills  relate  to  the  Army’s  mission  now  and  in  the  future. 

This  volume  contains  articles  written  by  Workshop  participants  and  recommendations  for 
future  work.  The  participants  included  highly  regarded  researchers  from  academia,  industry  and 
government  who  described  the  latest  research  in  training  critical  thinking  skills.  Military 
participants  in  the  Workshop  provided  evidence  of  the  importance  of  critical  thinking  skills  in 
battle  command. 

This  effort  was  briefed  to  Dean  of  Academics,  Army  Command  and  General  Staff 
College  (CGSC),  and  beneficiaries  of  Workshop  results  included  course  developers  and 
instructors  of  critical  thinking  in  the  CGSC  Intermediate  Level  Education  (ILE).  The  type  of 
collaboration  resulting  from  this  workshop  is  an  essential  step  in  the  development  of  new 
approaches  and  techniques  to  improve  critical  thinking  skills  in  the  Army. 

OTA  M.  SIMUTIS 
Technical  Director 
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Overview 


There  is  a  growing  interest  in  critical  thinking  skills  in  the  Army  training  community  as  a 
way  to  equip  officers  to  deal  with  the  greater  uncertainties  and  complexities  of  the  future  battle 
field,  rapid  technology  advances  and  the  explosion  of  instantly  available  communication  and 
information.  In  support  of  this  interest,  the  Army  Research  Institute’s  program  includes 
developing  methods  for  training  critical  thinking  skills  (CTS)  for  Battle  Command.  The  ARI 
Workshop  on  Critical  Thinking  Skills  for  Battle  Command,  described  in  this  Proceedings, 
provided  CTS  researchers  and  Army  training  developers  with  an  overview  of  the  latest  research 
in  CTS  training  and  frameworks  from  which  to  consider  methods  for  training  and  training 
evaluation.  It  also  provided  an  opportunity  for  participants  to  explore  together  the  challenges, 
problems,  and  solutions  and  the  implications  for  developing  and  training  CTS  in  Army  Officers. 


DESCRIPTION  OF  THE  WORKSHOP 

The  Workshop,  held  5-6  December  2000  at  Fort  Leavenworth,  was  conducted  by  the  U.S. 
Army  Research  Institute  with  contract  support  from  Caliber  Associates.  The  Workshop  was  part 
of  an  ARI  program  of  research  dealing  with  identifying  and  training  cognitive  and  critical 
thinking  skills.  It  brought  together  participants  with  a  variety  of  expertise  -  military  officers, 
instructors  in  Critical  Thinking  Skills,  and  researchers.  Because  of  this  mix,  the  group  was  able 
to  bring  different  perspectives  to  the  issues  that  were  discussed. 

The  objectives  of  the  Workshop  were  to:  provide  an  overview  of  current  research  in 
critical  thinking  (CT),  adult  learning,  and  CT  training;  provide  a  forum  for  identifying  and 
discussing  issues  related  to  training  CT  in  the  Army;  and  develop  recommendations  for  training 
CT  in  the  Army  and  directions  for  future  Research  and  Development. 

The  Workshop  had  two  main  sections.  First,  a  series  of  presentations  by  academic 
researchers  addressed  what  we  know  now  about  critical  thinking  and  training  critical  thinking. 
Next,  small  groups  discussed  the  implications  of  what  we  know  for  training  and  developing 
Critical  Thinking  Skills  in  Army  officers.  The  groups  also  developed  recommendations  for 
training  and  future  directions  for  Research  and  Development. 


WORKSHOP  PAPERS:  WHAT  DO  WE  KNOW  ABOUT  CRITICAL  THINKING  AND 
HOW  TO  TRAIN  IT? 

One  of  the  purposes  of  the  Workshop  was  to  provide  participants  with  a  tutorial  on  what 
educators,  psychologists,  and  philosophers  know  about  training  critical  thinking.  The  papers  deal 
with  the  major  issues  in  critical  thinking  and  training  CTS.  Critical  thinking  has  engendered  a 
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great  deal  of  interest  in  the  educational  and  training  communities.  A  search  of  the  Internet 
generated  219,601  citations  (web  pages)  involving  critical  thinking.  However,  there  is  little 
consensus  on  what  constitutes  critical  thinking  and  how  it  should  be  measured.  Workshop  papers 
give  an  overview  of  what  we  know  about  critical  thinking  and  provide  frameworks  to  integrate 
fragmented  theories  and  research.  The  first  presenter,  MG  (Ret)  Maggart,  sets  the  context  as  he 
discusses  the  need  for  critical  thinking  training  in  the  Army.  Diane  Halpem  then  gives  a  general 
overview  of  training  critical  thinking  and  Susan  Fischer  describes  a  framework  for 
conceptualizing  critical  thinking  and  its  training  that  would  be  useful  for  research  and  designing 
training.  Marvin  Cohen  provides  a  new  theory  of  critical  thinking  that  centers  on  dialog  as  a 
means  of  training  CT.  Daniel  Serfaty  describes  what  we  know  about  training  teams  in  critical 
thinking  and  Cohen  ends  with  a  description  of  work  that  is  being  done  in  developing  a  simulation 
of  critical  thinking  for  use  in  providing  intelligent  feedback  to  students. 

In  the  first  paper.  Critical  Thinking  in  the  21st  Century,  MG  Maggart  talks  about  the 
relationship  between  leadership  and  critical  thinking  and  the  need  for  critical  thinking  in  the 
military.  He  cautions  us  not  to  reduce  critical  thinking  to  a  process  and  describes  the  power  of 
mentoring  in  shaping  critical  thinking.  He  challenges  us  with  the  problem  of  how  to  inculcate 
critical  thinking  into  the  Army  culture  given  all  of  the  existing  demands  on  it.  Critical  thinking 
will  not  be  inculcated  into  the  Army  system  unless  there  is  a  dramatic  shift  in  the  existing  Army 
culture.  Not  only  must  the  Army  become  tolerant  of  critical  thinkers,  but  value  them  and  reward 
them. 

Next  Diane  Halpem  applies  cognitive  science  to  the  teaching  of  critical  thinking  in  her 
paper  Thinking  Critically  about  Critical  Thinking  and  presents  persuasive  arguments  for  why  we 
should  train  critical  thinking.  She  points  out  that  critical  thinking  training  is  predicated  on  two 
basic  assumptions:  (1)  there  are  identifiable  critical  thinking  skills  which  students  can  be  taught 
and  (2)  that  if  applied  will  improve  thinking.  Halpem  points  out  that  research  shows  that  students 
can  be  taught  to  become  better  thinkers  and  that  critical  thinking  skills  transfer  to  novel 
situations  when  we  teach  for  transfer.  She  proposes  a  model  for  critical  thinking  instruction  and  a 
taxonomy  for  organizing  college-level  critical  thinking  skills. 

Theoreticians  and  researchers  have  been  interested  in  critical  thinking  for  over  fifty  years. 
Despite  this  interest,  the  area  is  very  fragmented  and  there  is  not  a  dominant  theory  of  critical 
thinking.  Researchers  cannot  even  agree  on  a  definition  of  critical  thinking.  The  next  paper,  A 
Framework  for  Critical  Thinking  Research  and  Training  by  Susan  Fischer,  presents  a  framework 
for  critical  thinking  that  is  based  on  an  extensive  review  of  the  educational,  philosophical,  and 
psychological  literature.  The  purpose  of  the  framework  is  to  integrate  past  research  and  theories 
to  develop  a  model  that  would  be  useful  for  guiding  the  research  and  training  of  critical  thinking. 
This  model  provides  a  starting  point  for  the  Workshop  discussions  on  critical  thinking.  Fischer 
first  presents  an  overview  of  the  work  that  has  been  done  in  academia,  the  educational  arena,  and 
industry.  Then,  she  integrates  and  combines  others’  theories  to  propose  a  model  of  critical 
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thinking  that  provides  a  structure  for  the  aspects  of  critical  thinking  that  should  be  considered 
when  designing  and  conducting  training  in  critical  thinking.  The  results  of  the  application  of  this 
model  to  critical  thinking  in  the  domain  of  Army  command  and  control  are  described.  She  also 
provides  a  list  of  CTS  that  have  appeared  in  the  literature.  The  model  has  implications  for 
training  critical  thinking  and  these  are  discussed.  During  the  Workshop  participants  suggested  a 
number  of  changes  to  the  model  and  these  are  discussed. 

In  A  Three-part  Theory  of  Critical  thinking:  Dialogue,  Mental  Models,  and  Reliability, 
Marvin  Cohen  makes  the  case  that  critical  thinking  is  an  important  Army  battlefield  skill,  and 
that  its  importance  is  likely  to  increase.  But  it  is  necessary  to  consider  some  potential  problems 
that  may  arise  if  concepts  developed  in  academic  contexts  are  transferred  to  the  battlefield.  He 
presents  and  explains  a  theory  of  critical  thinking,  and  shows  how  it  addresses  some  of  the 
problems  inherent  in  Army  applications.  According  to  this  theory,  critical  thinking  skill  is 
exemplified  by  asking  questions  about  alternative  possibilities  in  order  to  achieve  some 
objective.  Asking  and  answering  questions  is  a  skill  of  dialogue.  Alternative  possibilities  are 
represented  by  mental  models.  A  process  of  questioning  mental  models  is  adopted  because  of  its 
reliability  for  achieving  the  purposes  of  the  participants  within  the  available  time.  This  paper 
also  addresses  questions  such  as:  Is  CT  consistent  with  tactical  battlefield  constraints?  Is  CT 
consistent  with  other  battlefield  skills:  Is  CT  appropriate  for  the  military  organizational 
structure?  Will  CT  fit  into  Army  training? 

In  his  paper  on  Critical  Thinking  in  Teams,  Darnel  Serfaty  reviews  the  state  of  knowledge 
on  team  performance  and  delineates  those  aspects  of  critical  thinking  and  adaptation 
strategies  which  are  unique  to  high-performing  teams.  The  paper  discusses  the  relationship 
between  the  constructs  of  shared  mental  models,  implicit  coordination,  and  teamwork-related 
critical  thinking  processes.  Hypotheses  are  proposed  on  the  factors  that  could  enhance  critical 
thinking  in  teams,  leading  to  the  timely  and  correct  decisions  to  communicate  information, 
coordinate  actions,  or  even  restructure  the  organization.  Potential  team  training  strategies  that 
focus  on  self-observation  and  adaptation  training  are  suggested.  Serfaty  concludes  that  while 
team  performance  research  has  seen  a  renewed  interest  in  the  last  few  years,  very  little  attention 
has  been  paid  to  the  specific  critical  thinking  processes  that  occur  in  command  teams. 

The  final  paper,  A  Simulation  Tool  for  Critical  Thinking  Training,  by  Marvin  S.  Cohen, 
Bryan  B.  Thompson,  &  Lokendra  Shastri,  describes  a  simulation  technology  that  combines  (1) 
the  ability  to  perform  rapid  recognitional  inferences  and  planning  within  a  large  (expert)  belief 
network,  (2)  human  limitations  on  computational  resources  and  attention,  and  (3)  critical 
thinking  skill.  This  tool  could  form  the  basis  for  the  development  of  an  adaptive  training  system, 
which  presents  realistic  military  decision  making  problems,  tracks  trainees’  responses,  and 
provides  appropriate  feedback  and  coaching  to  improve  both  recognitional  and  critical  thinking 
skills. 
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DISCUSSION  RESULTS:  CRITICAL  ISSUES  AND  RECOMMENDATIONS 

In  the  second  part  of  the  Workshop,  par.icipants  broke  into  small  groups  to  discuss  critical  issues 
in  CT  training  for  Army  officers  and  in  CT  research.  Following  is  a  summary  of  the 
recommendations  produced  by  those  discussions. 

RECOMMENDATIONS  FOR  IMPROVING  CT  TRAINING  IN  THE  ARMY 

■  Foster  an  Army  culture  that  values  CT 

■  Train  interpersonal  skills  for  the  application  of  CT 

■  Instructors  should  model  use  of  CTS  in  every  course 

■  Design  digital  capabilities  to  aid  and  facilitate  CT  and  use  these  capabilities  when 
training  CT 

■  Include  CT  training  in  every  course  that’s  taught  in  the  Army 

■  Train  for  transfer  of  training 

■  Explicitly  train  the  process  of  critical  thinking;  instructors  should  model  CT  in  all  of 
the  Army  officer  courses 

■  Capitalize  on  what  works  well  now. 

CRITICAL  ISSUES  FOR  FUTURE  RESEARCH  AND  DEVELOPMENT 

■  Development  of  valid  evaluation  methods  and  measures  of  CT 

■  Team  training  of  critical  thinking 

■  Individual  predispositions  to  CT.  What  conditions  elicit  CT?  What  conditions  stop 
CT? 

■  Simulations  to  explore  “what  if’  thinking  and  provide  intelligent  feedback. 
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Critical  Thinking  in  the  21st  Century 

MG  (RET)  Lon  E.  Maggart 


Thank  you  very  much  for  allowing  me  the  opportunity  to  address  this  very  distinguished 
group.  I  am  pleased  to  be  here  today  because  I  have  studied  and  thought  about  critical  thinking 
for  many  years  and  believe  it  is  not  only  an  important  topic,  but  also  one  that  is  essential  for  the 
success  of  the  Army  in  the  future.  I  am  also  very  pleased  that  ARI  has  seen  fit  to  address  this 
important  subject  today... it  couldn’t  have  been  better  timed  given  the  challenges  facing  the 
Army  today. 

Before  I  start,  let  me  say  that  the  short  talk  just  presented  by  Ed  Johnson  was  most 
eloquent.  It  is  almost  as  if  we  colluded  on  this  presentation.  He  stated  more  clearly  than  I  will  the 
significance  of  understanding  and  practicing  critical  thinking.  Let  me  also  say,  Ed,  that  every 
Army  officer  alive  today  has  read  Once  An  Eagle.  Each  can  categorize  him  or  herself  as  either 
Sam  Damon  or  Courtney  Massengale.  More  importantly,  each  can  categorize  their  fellow 
officers  likewise... and  do. 

I  think  the  short  answer  to  your  question  about  the  difference  between  these  two 
characters  is  that  Sam  Damon  took  care  of  people.  He  worked  his  way  to  the  top  with  care  and 
concern  for  people.  Massengale  got  to  the  top  on  the  backs  of  the  subordinates  and  peers  whom 
he  stomped  into  the  ground  as  he  ascended.  The  difference  is  that  Sam  Damon  made  a  positive 
difference  in  the  lives  of  all  those  with  whom  he  came  into  contact.  Massengale  may  have  been 
respected  for  his  guile,  but  he  was  never  admired  for  his  character.  The  lesson  for  us  all  from  this 
story  is  this:  take  care  of  people  and  make  their  experience  in  the  organization  positive. 
Leadership  success  is  transforming  people  to  be  better  than  they  were,  not  using  them  for 
personal  gain. 

Well,  on  with  the  lecture.  I  borrowed  the  following  passage  from  Dr.  Seuss. .  .by  the  way, 
it  is  one  that  General  Gordon  Sullivan  used  when  he  was  trying  to  get  the  Army  to  think  about 
digital  operations — captures  the  essence  of  my  message  today. 


CRITICAL  THINKING 


Think  left  and  think  right 
and  think  low  and  think  high 
Oh,  the  thinks  you  can  think  up 
if  you  only  try! 

Dr.  Seuss 
Figure  1 
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To  be  a  critical  thinker,  one  must  first  learn  how  to  think,  then  practice  it  without 
imposing  limitations  or  constraints.  I  am  not  so  sure  that  Dr.  Seuss  isn’t  worth  reading  as  adults 
for  the  mental  stimulation  it  provides  and  I  started  to  bring  a  number  of  copies  with  me  to  hand 
out  today  for  that  reason.  I  would  like  to  take  just  a  few  minutes  to  put  critical  thinking  in  the 
Army  into  some  historical  perspective. 

While  Dr.  Kathy  Quinkert  and  others  were  conducting  research  on  digital  operations  at 
Fort  Knox  in  the  late  1980’s,  a  general  understanding  of  the  potential  of  operating  digitally  did 
not  begin  until  the  early  1990’s  when  the  Army  began  to  critically  analyze  the  Gulf  War  and  the 
possibilities  for  future  war  that  were  enabled  with  the  new  Ml  A2  Abrams  tank.  The  inherent 
capabilities  of  this  new  tank  provided  a  glimpse  into  the  enormous  improvement  that  was 
possible  for  the  entire  combined  arms  team  because  every  member  of  the  team  could 
simultaneously  share  written  and  graphical  information  on  every  tactical  operation  whether 
deliberate  or  hasty.  The  possibilities  for  how  the  Army  could  fight  more  effectively  and 
efficiently  were  verified  during  the  first  digital  experiment  conducted  at  the  NTC  in  the  summer 
of  1994.  This  experiment  changed  forever,  at  least  intellectually,  notions  about  how  the  Army 
could  fight  and  set  the  stage  for  what  only  can  be  described  as  a  revolution  in  tactical 
thought... at  least  for  those  capable  of  critical  thought. 

It  was  precisely  in  the  months  following  this  experiment  that  new  and  exciting  skills — 
critical  thinking  skills — were  identified  that  leaders  needed  to  successfully  execute  high  tempo 
digital  operations.  Most  of  this  early  work  was  done  at  Fort  Knox  because  the  Ml  A2  tank  was 
the  genesis  of  digital  operations.  But  more  importantly,  Fort  Knox  was  the  agent  for  change  in 
the  Army’s  digitization  effort  in  large  part  because  the  leadership  at  Fort  Knox  had  the  vision  to 
see  the  possibilities  and  perhaps  most  importantly,  understood  the  power  in  using  simulations  to 
work  through  complex  problems. 

Interestingly,  those  working  most  closely  with  the  conversion  to  digital  operations 
discovered  that  these  new  skills  also  posed  several  unique  training  problems.  First,  coursework 
existed  for  virtually  none  of  the  new  skills  identified.  Secondly,  even  though  there  was  a  direct 
correlation  between  these  new  skills  and  using  the  digital  equipment  for  a  more  significant 
battlefield  payoff,  the  Army  had  no  real  strategy,  or  mechanism  to  train  soldiers  on  how  to  use 
digital  equipment  differently  from  existing  conventions  to  achieve  better  battlefield  effects.  The 
Army  knew  how  to  train  soldiers  to  operate  the  equipment,  that  is  turn  it  on  and  off,  access 
menus  and  so  on.,  .it  was  expert  on  such  training.  But  the  Army  did  not  know  how  to  train 
soldiers  to  use  the  equipment  to  conduct  tactical  operations  in  ways  that  demonstrated  the 
significantly  improved  battlefield  results  the  equipment  was  capable  of  producing.  The  major 
impediment  to  progress  then,  and  to  a  large  extent  now,  was  an  attitude  that  digital  tasks  and 
skills  are  the  same.  They  are  not.  The  biggest  hurdle  we  continue  to  confront  is  getting  people  to 
understand  that  one  can  fight  differently  using  digital  equipment. 
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For  example,  the  new  digital  equipment  made  it  possible  to  mass  fires  and  not  forces  on 
the  battlefield,  but  there  was  no  conceptual  tactical  framework  within  which  to  do  this  and 
certainly  no  methodology  to  teach  it.  Nor  was  there  any  concept  on  how  to  make  existing 
command  and  staff  actions  more  efficient.  In  fact,  some  of  the  great  minds  of  the  time  simply 
wanted  to  automate  the  existing  five-paragraph  field  order  as  a  way  to  address  the  problem.  So 
the  emphasis  was  put  on  what  the  Army  knew  how  to  do  best — teaching  soldiers  how  to  work 
the  machines.  Unfortunately,  little  or  no  thought  was  expended  on  how  to  make  the  machines 
give  soldiers  a  decided  advantage  on  the  battlefield.  The  first  point  I  want  to  make  here  is  that 
whatever  comes  from  this  conference  in  the  way  of  developing  strategies  for  critical  thinking, 

please  be  mindful  that  the  payoff  is  in  the  way  soldiers  and  leaders  can  convert  brainpower  into 
combat  power. 

The  second  point  I  want  to  make  is  that  the  Army  is  basically  a  process  driven 
organization,  more  interested  in  standardization  of  thought  than  in  divergent  thought.  Critical 
thinking  clearly  requires  divergent  thought,  so  as  you  discuss  critical  thinking  over  the  next  two 
days,  please  try  to  resist  the  urge  to  develop  another  military  decision  making  process  or  a 
command  and  staff  action  process.  If  you  do  so,  it  will  be  doomed  by  the  same  limitations  and 
restrictions  to  productive  thought  from  which  these  and  other  Army  processes  currently  suffer. 

To  that  point.  Figure  2  represents  my  notion  of  critical  thinking.  It  is  a  process  only  in 
that  it  involves  interaction  between  the  subject  areas  shown  on  the  periphery  and  probably  a  few 
we  haven’t  thought  of,  and  a  product  of  some  kind.  From  my  perspective,  critical  thinking  is  far 
too  complex  to  reduce  to  a  simple  algorithm  anyway.  But  again,  because  the  Army  is  process 
oriented,  the  urge  to  do  so  will  be  overwhelming.  This  is  a  caution  to  all  of  you!  If  you  describe 
critical  thinking  as  an  algorithm,  that  is  what  the  Army  will  teach  and  students  will  learn... but 
unfortunately  they  will  not  learn  how  to  think  critically! 


Figure  2 
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Once  the  world  and  subsequently  the  Army  moved  into  the  Information  Age,  life  became 
more  rather  than  less  complex  in  most  respects. 

Leaders  now  must  deal  with  an  entirely  new  set  of  intellectual,  cultural  and  equipment 
challenges  that  were  not  present  just  six  years  ago.  These  challenges  plus  the  advent  of  digital 
information  systems  that  allow  communications  at  rates  and  to  places  never  before  possible  and 
way  more  data  than  a  normal  human  can  deal  with,  all  require  substantial  changes  in  the  skills 
required  of  leaders  as  well.  Traditional  leadership  techniques  and  practices  simply  will  not 
suffice  in  the  months  and  years  to  come.  Leaders  must  therefore  be  able  to  think  on  their  feet, 
make  rapid  and  accurate  decisions,  take  the  initiative,  be  more  aware  of  their  capabilities  and 
adapt  instantly  to  rapidly  changing  even  chaotic  situations  using  divergent  thinking  to  process 

enormous  amounts  of  information  to  reach  an  acceptable  solution  that  will  deal  effectively  with 
the  circumstances. 

(One  of  my  aforementioned  colleagues  commented  on  the  thought  this  way:  “They  have 
to  understand  what  the  technology  enables  them  to  do  and  then  get  on  with  leading.  Part  of  the 
problem  is  that  many  people  want  to  be  the  operators  of  the  equipment  rather  than  users  of  the 
capability.  The  Army  needs  technically  capable  people,  but  not  in  the  combined  arms  leadership 
role.  There  also  needs  to  be  an  understanding  that  this  technology  will  empower  subordinates  to 
think  and  act  in  ways  never  before  possible.  But  to  achieve  that  which  is  possible,  the  Army 

can’t  continue  to  treat  NCOs  and  Enlisted  personnel  as  they  have  in  the  past  or  it  will  lose  the 
backbone.”) 

Figure  3  compares  traditional  leadership  skills  with  those  required  for  success  in  the  near 
future.  I  have  highlighted  the  skills  that  I  believe  require  critical  thinking  for  success. 


in  Addition  to  . . . 


intuition 

^  Versatility  &  Adaptability 
Vision  &  imagination 
*v  Innovative  Thinking 
f  Information  Age  Competence 
S  Sense  of  Humanity 
S  Creativity 

Conceptual  Thinking 


Traditional 
V  Skills 


*  Problem  Solving 
S  Organization 
S  Tactical  Proficiency 
s*  Physical  and  Mental  Stamina 
Decisiveness 
Work  Ethic 

S'  Self  Discipline  &  Confidence 
S'  Communication 

Judgment 


Figure  3 
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Meanwhile,  Figure  4  illustrates  several  leadership  challenges  for  this  century  and  the  skills 
necessary  to  cope  with  them.  I  think  it  is  fair  to  say  that  virtually  none  of  these  skills  are 
currently  in  the  curriculum  of  any  Army  school. 


j  Lead 

lership  for 

gli 

it  f:''' 

.  \  j 

Challenges 


-Organized  Complexity'.-. 
Contingencies 


Ethics 


Emerging  Cultural  Shifts 


Required  Skills 


Figure  4 

The  skills  highlighted  are  those  I  believe  require  critical  thinking  to  be  successful.  Please 
note  the  frequency  with  which  you  see  terms  like  adaptability,  flexibility  and  the  ability  to  adjust 
to  changing  circumstances.  Clearly  we  will  have  to  start  teaching  leaders  these  skills  very  early 
m  their  careers  if  we  hope  to  develop  senior  leaders  who  are  mentally  flexible.  The  entire  area  of 
leadership  is  in  flux  now  in  part  because  of  the  cultural,  interpersonal  and  organizational  changes 
necessary  to  cope  with  the  enormous  potential  of  information  age  technology. 

I  think  that  every  great  leader  I  have  known  or  read  about  was  able  to  coalesce  disparate, 
complex  thoughts,  ideas  and  concepts  into  a  single  expression  that  everyone  could  understand. 
Great  leaders  are  able  to  see  things  that  others  cannot.  All  of  them  not  only  see  the  big  picture; 
they  have  the  special  ability  to  explain  it  so  others  can  see  it  as  well.  And  lastly,  all  great  leaders 
believe  that  the  mind  is  the  most  powerful  weapon  in  their  arsenal.  All  great  leaders  spend  more 
time  thinking  than  talking... despite  what  you  normally  see  or  hear  around  the  Army. 

How  many  of  you  can  see  the  picture  in  the  middle  of  this  sea  of  black  dots  (Figure  5)? 
There  is  a  hunting  dog  there.  Do  you  all  see  it? 


9 


PERCEPTIONS 


A  random  dot  pattern? 
Figure  5 


The  point  I  want  to  make  is,  like  it  or  not,  your  brain  processes  what  it  sees  in  ways  that 
may  mislead  your  thought  processes. 

In  a  manner  of  speaking,  you  may  be  critically  thinking  about  the  wrong  thing.  Your 
brain  will  focus  on  the  big  picture,  then  try  to  improve  the  organization  of  the  material  it  sees. 
My  colleague  Dr.  Robert  Hubal  would  suggest  the  word  is  “impose”  not  “improve,”  but  in  either 
case,  your  brain  tries  to  make  sense  of  what  it  sees.  Context  is  important  in  helping  the  brain 
interpret  what  it  sees  so  you  can  be  misled  easily  if  you  don’t  understand  the  context.  The  lesson 
here  is  to  make  sure  that  you  have  the  correct  input  before  you  grind  through  some  critical 
thinking  process.  You  must  be  specific  about  your  focus.  Sometimes  you  will  need  to  see  the  big 
picture,  sometimes  the  details.  The  trick  (necessary  skill)  is  to  know  when  to  focus  on  which. 

We  have  been  talking  sometime  now  about  the  relationship  between  leadership  and 
critical  thinking  and  the  requirement  for  critical  thinking  skills.  The  real  question  is  how  do  we 
inculcate  critical  thinking  into  Army  today  given  all  the  existing  demands  on  it.  Figure  6 
represents  my  concept  of  how  to  accomplish  this.  This  diagram  is  roughly  equivalent  to  the 
current  three-legged  system  the  Army  uses  to  model  training:  individual  study,  schoolhouse 
instruction  and  training  conducted  in  units.  It  is  also  a  reflection  of  my  own  experience  over  30 
years.  I  attribute  whatever  critical  thinking  skills  I  possess  to  exposure  to  these  three  factors.  1 
think  that  a  combination  of  literature,  exposure  to  powerful  minds  and  the  ability  to  use 
simulations  for  creative  thinking  are  the  key  tenets  to  successfully  integrating  critical  thinking 
into  the  Army. 
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THE  ROAD  AHEAD 

llfM  LITERATURE  IftSS 


SIMULATION 


Figure  6 

At  the  pinnacle  of  the  triangle  is  literature.  I  am  not  talking  here  about  providing  to 
students  a  comprehensive  bibliography  or  a  suggested  reading  list.  Rather  I  am  talking  about 
giving  to  some  reputable  institution  like  ARI  the  requirement  to  review  all  pertinent  literature 
and  develop  a  compendium  of  information  that  best  captures  the  essence  of  critical  thinking.  I 
should  make  it  clear  here  that  I  am  not  talking  about  collecting  articles  and  papers  into  a  book. 
What  I  am  talking  about  is  collecting  sentences  and  paragraphs  from  existing  works  that 
precisely  get  at  the  essence  of  critical  thinking.  If  one  collected  the  short  historical  vignettes  the 
Army  sometimes  uses  in  its  field  manuals  into  a  single  volume  on  tactics,  this  is  close  to  the  idea 
I  have  in  mind.  I  believe  that  this  compendium  should  be  revised  annually  and  available  to 
everyone  on  the  Internet.  In  addition,  I  think  the  old  Excelnet,  or  something  like  it,  should  be 
revived  to  provide  an  open  forum  across  the  Army  for  people  to  discuss  critical  thinking. 

Do  any  of  you  remember  the  Excelnet?  I  didn’t  think  so.  The  Excelnet  was  established  in 
the  early  1 980’ s  by  a  friend  of  mine  named  Mike  Rodier,  now  deceased,  as  a  mechanism  for 
discussing  across  the  Army,  a  variety  of  subjects  generally  related  to  tactical  doctrine.  For 
example,  there  was  a  special  section  just  for  battalion  commanders  who  were  on  rotation  to  go  to 
the  NTC.  I  found  this,  as  did  many  of  my  peers,  to  be  a  most  useful  tool  as  there  were  interesting 
lessons  learned,  ideas,  tactics  and  techniques  and  procedures  discussed  on  this  net.  Everyone 
who  participated  was  free  to  provide  whatever  input  seemed  appropriate.  It  was  the  forerunner  to 
current  chat  rooms.  Occasionally  Mike  would  ask  a  participant  to  expand  a  particular  point  that 
he  or  she  had  discussed  on  the  net  into  an  article.  Mike  then  collected  and  published  these 
articles  in  a  volume  that  was  distributed  around  the  Army.  Unfortunately,  many  young  leaders 
used  the  Excelnet  to  vent  on  the  Army’s  senior  leadership  much  as  they  do  on  the  Internet  today 
and  it  was  disbanded  after  a  few  years.  I  believe  that  we  should  use  a  similar  process  to  discuss 
critical  thinking,  publish  selected  works  and  make  them  available  across  the  Army. 

I  believe  that  direct  contact  with  unique  individuals  of  powerful  intellect,  creative  minds 
and  interesting  life  experiences  is  essential  in  expanding  ones  own  power  of  critical  thinking.  It 
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is  like  the  Dr.  Seuss  example  earlier;  you  just  don’t  know  what  you  don’t  know  until  someone 
explains  something  new  to  you.  My  exposure  to  great  minds  like  Jack  Woodmansee,  Gordon 
Sullivan,  Dee  Hock— who  thought  up  and  put  into  being  the  VISA  credit  card  company— Peter 
Senge,  Margaret  Wheatley,  Mike  Malone  and  others  certainly  expanded  my  ability  to  think.  And 
I  might  add  here  that  powerful  minds  aren’t  the  sole  province  of  older,  established  intellectuals. 
Some  of  the  most  powerful  influences  on  my  intellect  came  from  people  younger  than  1. 1 
learned  some  of  the  most  important  things  imaginable  from  the  likes  of  Major  Art  DeGroat  and 
Colonel  Greg  Fontenot  sitting  right  here  in  this  room.  Mike  Shaler  remains  a  trusted  agent 
against  whom  I  can  bounce  ideas.  Colonel  Jeanette  James,  an  Army  nurse,  taught  me  things  I  had 
never  even  thought  about  before  and  along  the  way  a  pretty  healthy  way  to  think  about  gender 
issues.  Dr.  Kathy  Quinkert  has  been  a  valued  colleague  advising  me  on  a  range  of  topics  from 
training  to  human  interaction.  To  this  day  I  send  almost  every  significant  thing  I  write  to  each  of 
these  people  for  their  opinion.  If  one  isn’t  concerned  about  another’s  age,  rank  or  gender,  it  is 
truly  amazing  what  one  can  learn! 

Now  clearly  the  Army  can’t  afford  to  send  a  parade  of  famous,  high  dollar  personalities 
around  to  every  schoolhouse  and  post.  But  the  Army  certainly  can  afford  to  hire  some  of  these 
folks  to  make  videotapes,  write  papers  or  tape  lectures  that  can  be  sent  around  in  this  fashion. 

The  Army  can  also  afford  to  link  special  meetings  and  conferences  with  thirty  minutes  or  an  hour 
of  discourse  by  some  noted  intellectual.  And  there  are  plenty  of  intellectuals  who  can  entertain  as 
they  transmit  information. 

I  remain  a  tremendous  fan  of  mentoring.  Not  the  designated  kind  of  mentoring  the  Army 
uses,  but  the  kind  that  forms  from  mutual  interest  in  a  subject  or  perhaps  because  of  an  emotional 
bond  between  the  mentor  and  mentee.  In  fact  I  would  venture  that  it  is  possible  to  pass  more 
information  in  a  shorter  period  of  time  using  mentoring  than  in  any  other  single  method  of 
teaching.  If  you  think  of  the  ability  of  a  mentor  to  take  all  the  time  necessary  to  explain  complex 
thoughts  in  depth,  to  give  personal  examples  and  to  answer  questions  virtually  any  time,  day  or 
night,  it  is  easy  to  understand  the  power  of  mentoring.  As  an  aside,  there  probably  is  no  question 
off  limits  between  a  mentor  and  a  mentee,  so  sensitive  subjects  can  be  discussed  that  would 
otherwise  be  impossible. 

In  any  event,  the  Army  must  take  special  pains  to  ensure  exposure  to  a  wide  and  diverse 
range  of  powerful  minds.  This  includes  individuals  who  espouse  views  contrary  to  those  valued 
by  the  Army  or  even  the  Federal  Government  for  that  matter. 

My  last  point  on  this  subject  is  a  plea  for  the  Army  to  relax  its  too  stringent  restrictions 
on  the  personal  use  of  government  computers  and  access  to  the  Internet.  My  view  has  always 
been  that  the  Army  needed  to  promote  divergent  thought  amongst  it  soldiers  and  leaders  not 
restrict  it.  What  better  way  than  to  use  email  for  interaction  between  people?  And  it  is  clear  to 
me  that  the  enormous  benefit  gained  from  people  internetting  thoughts,  ideas  and  experiences  far 
exceeds  any  problem  that  results  from  “abusing”  the  system.  If  one  is  found  to  be  frequenting 
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pom  sites  or  using  the  system  to  run  a  personal  business,  the  organization  has  a  different  problem 
than  “misuse”  of  computers. 

The  final  leg  of  my  triangle  concerns  the  use  of  simulations  as  a  tool  for  the  intellectually 
curious  to  consider  advanced  thoughts  in  a  dynamic  environment  against  a  range  of  possible 
constructs. 

For  this  to  be  a  reasonable  course  of  action,  the  Army  must  invest  in  research  and 
development  into  the  best  kind  of  simulation  and/or  model  for  this  purpose.  It  may  well  be 
possible  to  adapt  an  existing  simulation  like  Janus,  but  in  any  event,  some  energy  needs  to  be 
expended  in  this  area.  F or  this  notion  to  work  simulations  chosen  for  this  purpose  must  be 
accessible  to  Army  and  DOD  civilians  on  every  large  post  in  the  Army  who  want  to  use  it.  There 
must  be  minimal  restrictions  on  what  the  simulation  is  used  to  investigate  and  the  results  of  these 
simulations  must  be  made  available  across  the  Army  through  some  mechanism  like  Excelnet. 

And  since  we  are  dealing  with  ideas  and  thinking,  the  process  is  not  so  important  as  the  product. 
In  other  words,  it  will  not  be  necessary  to  produce  statistically  analyzed  data  for  the  product  of 
these  simulations  to  be  significant  or  to  be  disseminated  across  the  Army.  Wild  speculation  will 
be  acceptable  as  a  by-product  of  these  simulations.  The  bottom  line  is  that  simulations  that 
permit  “out  of  the  box”  thinking  and  experimentation  must  be  accessible  to  all  who  have  an 
interest  in  engaging  in  this  kind  of  activity  if  we  are  going  to  push  the  frontier  on  critical 
thinking. 

To  illustrate  this  point,  let  me  give  you  an  example  of  something  we  did  at  Fort  Knox  five 
years  ago.  We  assembled  a  group  of  very  smart  majors  and  captains  who  planned  and  executed 
an  offensive  operation  with  a  fully  digitized  mounted  brigade  against  a  modernized  opposing 
force  defending  a  piece  of  terrain.  We  postulated  the  capability  of  such  a  force  and  then  modified 
the  variables  in  the  Combat  Developments  Janus  simulation  to  accommodate  these  parameters. 

I  brought  in  all  of  our  foreign  liaison  officers  to  serve  as  the  command  group  for  the 
opposing  force.  We  then  executed  an  operation  in  which  we  infiltrated  the  entire  brigade  through 
the  defensive  lines  of  the  opposing  force  without  being  detected.  We  destroyed  the  bulk  of  the 
opposing  force  by  maneuver,  massing  the  effects  of  fires  without  massing  forces  except  when 
absolutely  necessary.  We  also  suffered  few  casualties  and  reached  a  decision  favorable  to  the 
friendly  force  considerably  more  rapidly  than  would  have  been  possible  with  a  convention  force 
of  similar  size  conducting  an  operation  appropriate  for  such  an  organization. 

This  exercise  permitted  us  to  conduct  an  operation  at  a  time  in  the  future  against  an 
opposing  force  that  did  not  exist  with  a  friendly  force  organized  in  a  way  that  did  not  exist, 
equipped  with  digital  systems  that  did  not  exist  and  using  tactics  that  did  not  exist  at  the  time  we 
conducted  the  experiment. 

We  derived  a  number  of  compelling  insights  into  how  battles  could  be  fought  if  a  unit 
was  equipped  in  a  very  specific  way.  This  excursion  allowed  us  to  think  critically  about  future 
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operations  in  ways  that  never  would  have  been  possible  by  conjuring  in  classroom.  And  we  did 
so  in  the  course  of  an  afternoon,  not  after  weeks  of  intellectual  exploration.  Curiously,  it 
occurred  to  me  that  this  was  precisely  the  way  we  should  be  establishing  our  acquisition 
requirements  documents  and  evaluating  operations  and  organization  concepts  as  well.  It  was  also 
one  of  my  earliest  insights  to  the  power  of  using  simulations  to  promote  critical  thinking. 

Let  me  now  illustrate  a  practical  implementation  of  critical  thinking.  I  sent  the  next  series 
of  slides  to  General  Sullivan  when  he  was  the  CSA  as  a  way  to  help  him  think  through  the 
complex  problem  of  integrating  disparate  Army  processes  in  a  different  way.  I  sent  him  these 
slides  for  two  reasons.  First  to  define  the  problem  that  1  thought  he  had  in  trying  to  run  an  Army 
who’s  major  operating  systems  at  best  were  uncoordinated  and  at  worst  dysfunctional.  Secondly, 
I  wanted  to  illustrate  how  critically  thinking  through  the  problem  could  offer  some  tangible 
solutions. 

I  tried  to  define  the  problem  by  illustrating  how  the  major  Army  systems  currently 
worked.  Notwithstanding  the  “How  the  Army  Runs”  (HOWTAR)  chart  made  famous  by  General 
Max  Thurman,  in  reality  all  of  the  major  Army  systems  were  stovepipes.  Note  than  there  was  no 
linkage  between  any  of  them.  Anomalies  had  to  be  managed  manually  by  the  DA  Staff.  This  fact 
created  massive  inefficiencies  in  the  system  that  impacted  greatly  on  the  operating  forces  in  the 
field.  As  a  case  in  point,  I  tried  unsuccessfully  for  years  as  the  Deputy  Commanding  General  and 
later  as  the  Commandant  of  the  Armor  School  to  convince  TRADOC  that  there  was  no  actual 
correlation  between  dollars,  people  and  the  Program  of  Instruction  (POI)  even  though  my  budget 
and  manpower  were  managed  as  if  there  was. 

I  think  the  Army  recognized  there  were  inefficiencies  in  the  system  and  at  least  on  a 
global  scale,  tried  to  fix  them  as  illustrated  in  Figure  7.  Note  that  the  fixes  were  merely  band-aids 
and  none,  save  for  Louisiana  Maneuvers  and  the  Joint  Venture  efforts,  even  made  the  attempt  to 
link  all  of  the  DOTLMS.  The  problem  with  band-aid  solutions  is  that  they  are  temporary  fixes, 
they  create  a  new  set  of  problems  that  exist  outside  the  established  processes  and  they  continue 
to  drain  resources  long  past  when  they  are  useful. 
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THE  ARMY  “FIX” 


Figure  7 

I  suggested  a  solution  that  used  existing  processes  to  integrate  the  stovepipes  in  a  way 
that  resulted  in  a  rope  rather  than  a  series  of  unrelated  pieces  of  string.  Note  that  critical  thinking 
is  the  oil  that  lubricates  the  process  (Figure  8). 
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THE  REAL  SOLUTION 
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THINKING 


INTEGRATED 
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Figure  8 


Our  job  at  this  conference  is  to  figure  out  how  to  weave  the  various  components  of 
critical  thinking  into  a  process  that  provides  a  mechanism  for  the  Army  to  integrate  critical 
thinking  into  its  day-to-day  activities  in  a  positive,  non-disruptive  way  as  illustrated  by  Figure  9. 

There  is  one  key  point  to  this  process  that  might  not  be  immediately  apparent  from  the 
figure.  Note  that  on  the  left  side  I  have  the  word  “divergent”  and  on  the  right  side  the  word 
“convergent.”  The  concept  I  am  trying  to  convey  is  that  the  process  of  critical  thinking  employs 
a  number  of  different  divergent  thinking  skills,  that  is,  creativity,  innovation  and  so  on,  to  permit 
the  widest  range  of  thoughts  to  be  considered  as  the  basis  for  developing  an  informed  decision. 
The  process  must  then  have  some  mechanism  to  converge  this  collection  of  thoughts  into  a 
decision  or  conclusion. 

About  two  years  ago,  I  gave  a  lecture  at  Fort  Benning  for  Scott  Graham  on  Situational 
Awareness.  In  conjunction  with  the  lecture,  Dr.  Hubal  and  I  prepared  a  paper  in  which  we 
developed  a  situational  awareness  model  partially  as  a  way  to  describe  the  function  of  intuition 
in  military  decision-making.  As  I  thought  about  how  to  explain  my  view  on  critical  thinking  for 
this  group,  it  occurred  to  me  that  the  situational  awareness  model  was  in  fact  also  descriptive  of 
this  process  as  well.  In  fact,  while  the  model  was  designed  as  specific  to  situational  awareness, 
much  of  it  may  well  be  generally  applicable  to  critical  thinking  processes. 

If  you  will  refer  to  Figure  10, 1  think  you  quickly  will  see  how  we  adapted  the  model  for 
critical  thinking.  Note  that  automatic  and  conscious  decision  processes  are  central  to  the  Critical 
Thinking  Model.  Both  of  these  processes  involve  what  we  describe  as  native  skills  such  as 
imagination,  visualization,  innovation  and  so  on.  We  believe  it  is  possible  to  learn  and/or  refine 
these  skills,  although  we  clearly  provide  for  innate  talent  by  identifying  the  bulk  of  these  skills 
under  a  category  entitled  “Native  Skills.” 
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CRITICAL  THINKING  MODEL 
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Figure  10 


We  believe  that  the  conscious  decision  process  involves  the  formulation  of  mental 
representations  that  are  achieved  using  visualization  and  known  facts  about  the  battlefield.  We 
used  the  expression  '‘analogical  engine’  to  refer  to  the  automatic  or  subconscious  decision 
processes  that  use  one  or  several  native  skills  in  concert  with  established  well-learned  patterns  or 
procedures.  We  believe  that  battlefield  decisions  involve  a  combination  of  both  conscious  and 
automatic  processes.  Both  processes  are  fueled,  sometimes  instantly,  sometimes  as  a  product  of 
deliberate  reasoning  over  an  expanded  period  of  time,  by  external  inputs.  These  inputs  take  the 
form  of  perceptual  and  motor  skills  acquired  through  experience  and  education  and  episodic  or 
mission  specific  information  and  data  acquired  spontaneously  while  the  event  is  underway  from 
a  variety  of  sources.  Both  automatic  and  conscious  decision  processes  can  be  enhanced  by  the 
way  technology  is  used  to  organize,  evaluate,  monitor  and  portray  information. 

Curiously,  the  Army  appears  to  have  a  better  understanding  of  critical  thinking  than  it 
does  about  the  kind  of  environment  that  is  necessary  to  develop  it.  Figure  1 1  represents  an 
unscientific  sample  of  views  around  the  Army  on  the  subject  of  critical  thinking. 
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Some  Thoughts  on  Critical  Thinking 
From  the  Field 


“The  down  side  to  critical  thinking  in  a  military 
culture  that  does  not  embrace  and  value  it  is  a 
learned  sense  of  frustration  over  the  lack  of 
interest  to  do  something  with  an  unconventional 
perspective  or  novel  proposal  or  idea.  Most  of 
our  greatest  cynical  officers  are  frustrated 
critical  thinkers  who  have  given  up  that  the 
institution  will  ever  listen!” 

“The  Army  has  more  critical  thinkers  than  it 
benefits  from  because  senior  officers  do  not 
encourage,  appreciate  or  reward  their  different 
ideas.” 

“Critical  thinkers  unconstrain  themselves  from 
the  situation  to  ‘see’  possible  solutions — then  go 
back  and  modify  conditions  to  make  their  system 
WORK!” 

“Critical  thinkers  are  more  outcome/results 
oriented  than  non-critical  thinkers  who  are 
content  with  the  idea  that  if  they  follow 
established  procedures,  then  the  desired  effect 
will  result.” 


Figure ll 

It  is  clear  from  these  comments  that  there  is  a  reasonable  understanding  of  critical 
thinking  in  the  field  and  more  importantly,  there  are  people  who  are  willing  to  use  critical 
thinking  skills...  if  only  they  were  permitted  to  do  so.  It  is  equally  clear  that  the  current  Army 
culture  neither  encourages  nor  is  tolerant  of  those  who  think  outside  the  box.  The  point  is  that 
even  if  we  were  to  design  a  functional  program  exactly  as  I  described  earlier,  critical  thinking 
will  not  become  inculcated  into  the  system  until  there  is  a  dramatic  shift  in  the  existing  Army 
culture.  As  a  minimum  this  shift  must  include  tolerance  for  divergent  thought,  but  hopefully,  it 

will  swing  more  toward  the  positive  end  where  those  capable  of  such  thought  become  valued 
members  of  the  Army. 

There  is  more  to  the  problem  of  expanding  critical  thinking  in  the  Army  than  developing 
a  curriculum  and  creating  an  environment  that  encourages  or  at  least  is  tolerant  of  it.  The  Army 
must  underwrite  tolerance  for  intellectual  diversity  and  be  flexible  in  accepting  alternative 
solutions.  Most  importantly,  the  Army  must  underwrite  the  decision  by  men  and  women  of 
character  to  take  a  stand  against  established  but  dysfunctional  or  illogical  norms.  Senior  leaders, 
in  particular,  need  to  understand  that  there  are  valid  perspectives,  viewpoints  and  opinions  that 
differ  from  their  own  that  have  equal  or  greater  merit. 
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Soldiers  and  leaders  need  to  understand  that  it  is  hard  to  be  a  critical  thinker  if  the  most 
important  thing  is  another  promotion  and  the  ideal  assignment.  Those  who  worry  about  making 
career-ending  mistakes  will  never  be  satisfactoiy  critical  thinkers. 

So,  in  conclusion,  ladies  and  gentlemen,  (Figure  12)  describing  the  way  ahead  is 
relatively  simple.  We  must  attack  each  of  these  three  areas  simultaneously.  First,  we  need  to 
change  the  Army  culture  so  that  both  critical  thinking  and  critical  thinkers  become  acceptable 
and  important.  This  can  only  happen  if  the  Army’s  senior  leadership  creates  the  proper 
environment  by  adjusting  the  various  selection  processes  to  recognize,  promote  and  school 
individuals  with  these  skills;  by  assignments  that  reward  critical  thinking  skills  and  lastly;  by 
modifying  the  officer  efficiency  report  so  that  individuals  who  possess  critical  thinking  skills  can 
be  identified,  managed  and  more  importantly,  rewarded.  Secondly,  we  must  decide  on  a 
theoretical  construct  or  model  with  which  everyone  can  live  that  describes  critical  thinking.  You 
can  use  the  one  Rob  Hubal  and  1  developed,  expand  the  one  Susan  Fischer  offered  or  create 
something  entirely  new.  In  any  case,  the  model  selected  then  becomes  the  basis  for  developing 
strategies  for  thinking  about,  teaching  and  applying  critical  thinking  skills.  Lastly,  we  must 
define  some  process  for  inculcating  critical  thinking  into  our  school  systems,  units  and  of  course 
our  leaders  and  soldiers  in  a  fashion  that  does  not  disrupt  an  already  overtaxed  system. 


CONCLUSION 


■  CULTURAL  CHANGE. ..MAKE  CRITICAL 
THINKING  IMPORTANT  FOR 

SUCCESS... SELECTIONS,  ASSIGNMENTS,  OERS 

■  CHOOSE  A  MODEL.. .THE  CRITICAL  THINKING 
MODEL 

■  DETERMINE  THE  PROCESS... THE  TRIANGLE 

Figure  12 

I  use  Figure  13  in  part  because  it  best  describes  the  mindset  that  will  be 
make  critical  thinking  a  reality  in  the  Army.  NO  MISSION  TOO  DIFFICULT, 
TOO  GREAT,  DUTY  FIRST.  Thank  again  for  having  me  today.  It  was  a  great 
honor  for  me. 


necessary  to 
NO  SACRIFICE 
pleasure  and 


20 


NO  M  ESDN  TOO  D3PFEULT 
NO  SACRIFICE  TOO  GREAT 


DUTY  FIRST  ! 


Figure  13 
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Thinking  Critically  About  Critical  Thinking: 
Lessons  From  Cognitive  Psychology 

Diane  F.  Halpern 

California  State  University,  San  Bernardino 

Let  me  give  you  some  scary  facts  about  the  American  public.  First,  a  large  percentage 
start  everyday  by  reading  their  horoscope  and  believe  that  it  is  so  often  correct  that  it's  as  though 
it  was  written  especially  for  them;  they  phone  their  personal  psychic,  at  a  cost  that  many  cannot 
afford,  for  advice  on  matters  that  range  from  how  to  invest  their  money  to  whether  or  not  a  loved 
one  should  be  disconnected  from  life  support  systems;  they  spend  huge  sums  of  money  on  a 
variety  of  remedies  for  which  there  is  no  evidence  that  they  work  or  are  even  safe  to  take— 
sometimes  with  disastrous  results.  In  a  survey  of  college  students,  more  than  99  percent 
expressed  belief  in  at  least  one  of  the  following:  channeling,  clairvoyance,  precognition, 
telepathy,  psychic  surgery,  psychic  healing,  healing  crystals,  psychokinesis,  astral  travel, 
levitation,  the  Bermuda  triangle  mystery,  unidentified  flying  objects  (UFOs),  plant 
consciousness,  auras,  and  ghosts  (Messer  &  Griggs,  1989). 

Of  course,  there  are  a  variety  of  the  alien  abductions,  the  unconscionable  charlatans  who 
make  money  by  helping  grieving  people  keep  in  touch  with  their  deceased  loved  ones,  and  many 
more.  If  you’re  not  worried  yet,  consider  that  political  candidates  and  public  policies  are 
marketed  to  the  general  public  with  the  same  techniques  that  are  used  to  persuade  us  to  buy  the 
“toothpaste  with  sex  appeal.”  Let’s  add  to  this  list  the  so-called  “historical  societies”  that  deny  or 
minimize  the  atrocities  of  the  Holocaust  and  related  groups  that  explain  that  most  slaves  in  the 
United  States  enjoyed  their  protected  status.  Arguments  in  support  of  the  second  statement 
sometimes  take  the  form  that  slaves  were  valuable  property,  and  few  would  be  willing  to  mistreat 
or  damage  their  own  valuable  property.  Therefore,  most  slaves  must  have  been  treated  well. 

There  is  an  embedded  and  seductive  logic,  albeit  a  flawed  one,  that  is  used  to  explain  a  wide 
range  of  beliefs.  Of  course,  there  is  also  rationalization— the  hunting  for  reasons,  even  weak 
ones,  to  support  a  preconceived  conclusion.  I  probably  should  also  mention  the  flat  earth  society, 
a  society  based  on  the  belief  that,  you  guessed  it,  the  earth  is  flat. 

Given  all  of  these  examples,  it  is  not  surprising  that  many  colleges  in  the  United  States 
and  other  places  throughout  the  world  now  require  all  students  to  take  a  course  in  "critical 
thinking”  as  part  of  their  general  education  program.  Although  there  has  been  considerable 
disagreement  over  who  should  teach  such  courses,  whether  they  should  be  "stand-alone"  generic 
courses,  and  what  sorts  of  skills  students  should  be  learning  in  these  courses,  there  is  virtually  no 
disagreement  over  the  need  to  help  American  students  improve  how  they  think.  Both  George 
Bush,  the  elder,  and  Bill  Clinton,  when  he  was  governor,  supported  the  national  education  goal 
for  higher  education  that  declared  that  it  was  a  national  priority  to  enhance  critical  thinking  in 
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college  students,  although  I  might  add  that  this  national  priority  was  never  funded.  It  was  a  great 
idea  that  both  Democrats  and  Republicans  agreed  upon,  until  it  was  time  to  pay  for  it,  causing 
some  to  cynically  wonder  if  politicians  really  want  a  thinking  public. 

In  his  award-winning  book,  Earl  Hunt  (1995)  examined  the  skills  that  will  be  needed  by 
our  workforce  in  the  early  decades  of  this  century  and  asks,  “Will  we  be  smart  enough?”  The 
way  we  answer  this  question  will  determine  the  quality  of  life  that  we  and  our  children  can 
expect  to  live,  as  well  as  the  future  of  our  country  and  our  planet.  The  most  important  reason  for 
making  the  enhancement  of  critical  thinking  skills  the  primary  objective  of  education  is  that  the 
world  is  changing  at  an  accelerating  rate.  The  workforce  is  one  critical  place  where  we  can 
witness  the  dizzying  pace  of  change.  There  is  an  increased  demand  for  a  new  type  of  worker — 
with  a  new  job  category,  which  has  been  dubbed  the  “knowledge  worker”  or  the  “symbol 
analyst”  (a  term  that  is  used  by  the  United  States  Secretary  of  Labor),  to  describe  someone  who 
can  carry  out  multi-step  operations,  manipulate  abstract  and  complex  symbols  and  ideas,  acquire 
new  information  efficiently,  and  remain  flexible  enough  to  recognize  the  need  for  continuing 
change  and  for  new  paradigms  for  life-long  learning. 

Workers  in  almost  every  job  category  can  expect  to  face  novel  problems  because  the 
nature  of  the  workplace  is  changing  repeatedly.  Familiar  responses  no  longer  work  and  even 
newly  acquired  ones  won’t  work  for  long.  The  information  explosion  is  yet  another  reason  why 
we  need  to  provide  specific  instruction  in  thinking.  We  now  have  an  incredible  wealth  of 
information  available,  quite  literally  at  our  fingertips,  via  the  internet  and  other  remote  services 
with  only  a  few  minutes  of  "search  time"  on  the  computer.  The  sheer  quantity  of  data  that  is 
available  is  overwhelming.  Relevant,  credible  information  has  to  be  selected,  interpreted, 
digested,  evaluated,  learned,  and  applied  or  it  is  of  no  more  use  on  a  computer  screen  than  it  is  on 
a  distant  library  shelf.  If  we  cannot  think  intelligently  about  the  myriad  of  issues  that  confront 
us,  then  we  are  in  danger  of  having  all  of  the  answers,  but  still  not  knowing  what  they  mean. 

The  twin  abilities  of  knowing  how  to  learn  and  knowing  how  to  think  clearly  about  the  rapidly 
proliferating  information  that  they  will  be  required  to  deal  with  will  provide  the  best  education 
for  citizens  of  the  21st  century. 

Let  me  explain  what  I  mean  by  the  term  "critical  thinking"  and  suggest  that  we  use  this  as 
a  working  definition: 

The  term  critical  thinking  is  the  use  of  those  cognitive  skills  or  strategies  that  increase 
the  probability  of  a  desirable  outcome.  It  is  purposeful,  reasoned,  and  goal-directed.  It  is  the 
kind  of  thinking  involved  in  solving  problems,  formulating  inferences,  calculating  likelihoods, 
and  making  decisions.  Critical  thinkers  use  these  skills  appropriately,  without  prompting,  and 
usually  with  conscious  intent,  in  a  variety  of  settings.  That  is,  they  are  predisposed  to  think 
critically.  When  we  think  critically,  we  are  evaluating  the  outcomes  of  our  thought  processes — 
how  good  a  decision  is  or  how  well  a  problem  is  solved.  Critical  thinking  also  involves 
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evaluating  the  thinking  process— the  reasoning  that  went  into  the  conclusion  we’ve  arrived  at  or 
the  kinds  of  factors  considered  in  making  a  decision.  (Halpem,  1 996,  1 998) 

In  the  term  “critical  thinking,”  the  word  “critical”  is  not  meant  to  imply  “finding  fault,” 
as  it  might  be  used  in  a  pejorative  way  to  describe  someone  who  is  always  making  negative 
comments.  It  is  used  instead  in  the  sense  of  “critical”  that  involves  evaluation  or  judgment, 
ideally  with  the  goal  of  providing  useful  and  accurate  feedback  that  serves  to  improve  the 
thinking  process. 

Here  is  a  list  of  generic  skills  that  can  be  important  in  many  situations: 

A  critical  thinker  can: 

■  Recognize  that  a  problem  exists 

■  Develop  an  orderly  planful  approach  so  that  tasks  are  prioritized  and  problems  are 
recognized  as  differing  with  regard  to  how  serious  and  urgent  they  are 

■  Use  the  metacognitive  knowledge  that  allows  novices  to  monitor  their  own 
performance  and  to  decide  when  additional  help  is  needed 

■  Develop  an  openness  to  a  variety  of  solutions,  even  novel  ones 

■  Generate  a  reasoned  method  for  selecting  among  several  possible  courses  of  actions 

■  Give  reasons  for  choices  as  well  as  varying  the  style  and  amount  of  detail  in 
explanations  depending  on  who  is  receiving  the  information 

■  Recall  relevant  information  when  it  is  needed 

■  Use  skills  for  learning  new  techniques  efficiently  and  relating  new  knowledge  to 
information  that  was  previously  learned 

■  Use  numerical  information,  including  the  ability  to  think  probabilistically  and  express 
thoughts  numerically 

■  Understand  basic  research  principles 

■  Demonstrate  an  advanced  ability  to  read  and  write  complex  prose 

■  Present  a  coherent  and  persuasive  argument  on  a  controversial,  contemporary  topic 

■  Provide  complex  instructions  in  language  that  is  appropriate  for  the  audience 

■  Use  matrices  and  other  diagrams  for  communication 

■  Synthesize  information  from  a  variety  of  sources 

■  Determine  credibility  and  use  this  information  in  formulating  and  communicating 

decisions.  ' 


Critical  thinking  instruction  is  predicated  on  two  assumptions:  (1)  that  there  are  clearly 
identifiable  and  definable  thinking  skills  which  students  can  be  taught  to  recognize  and  apply 
appropriately,  and  (2)  if  recognized  and  applied,  the  students  will  be  more  effective  thinkers. 

I  can  probably  guess  what  you  are  thinking.  This  is  a  great  list  of  critical  thinking  skills 
and  you  could  easily  add  more,  but  is  there  good  evidence  that  adults  will  improve  in  these  skill 
areas  in  ways  that  transfer  to  novel  situations  and  endure  beyond  the  term  of  instruction  when 
they  are  specifically  taught  these  skills.  First,  there  is  no  reason  to  believe  that  the  ability  to  think 
critically  can’t  be  improved.  Most  students  improve  in  their  ability  to  write  after  taking 
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coursework  in  writing;  similarly,  most  students  become  better  at  oral  communication  after 
appropriate  instruction,  and  so  on  for  math  and  other  cognitive  skill  areas.  We  now  have  a 
sufficiently  large  body  of  research  that  shows  that  it  is  possible  to  teach  in  ways  that  help 
students  become  better  thinkers  and  that  these  skills  transfer  to  novel  situations  when  we  teach 
specifically  for  transfer. 


TYPES  OF  EVIDENCE 

■  “Blind”  evaluations  of  programs  designed  to  enhance  thinking  skills  (e.g.,  the 
Venezuela  project;  Hermstein,  Nickerson,  de  Sanchez,  &  Swets,  1986) 

■  Student  self  reports  (weak  evidence,  but  students  believe  that  they  have  improved- 
Block,  1985) 

■  Gains  in  cognitive  growth  and  development  (e.g.,  Piagetian  tasks  that  measure 
cognitive  stages;  Fox,  Marsh,  &  Crandall,  1983;  Kosonen  &  Winne,  1995) 

■  More  expert-like  mental  representations  following  instruction  (relative  to  control 
groups;  Shoenfeld  &  Hermann,  1982) 

■  Tests  of  cognitive  skills  (e.g.,  standardized  tests  for  critical  thinking;  Facione,  1991; 
Me  Bride,  1996) 

■  Spontaneous  and  uncued  transfer  (e.g.,  call  students  at  home  months  after  the  class  is 
completed  under  the  guise  of  a  survey;  Fong,  Krantz,  &  Nisbett,  1993) 

■  Inductive  reasoning  tasks  were  taught  to  college  students  using  realistic  scenarios 
from  many  different  domains.  The  authors  conclude  that  critical  thinking  is  “a  skill” 
and  that  “it  is  transferable”  (Jepson,  Krantz,  &  Nisbett,  1 993,  p.  82). 

Traditionally,  instruction  in  how  to  think  has  been  a  neglected  component  in  American 
education.  Students  were  more  often  taught  what  to  think  than  how  to  think,  or  in  a  jargon  you 
may  be  more  familiar  with,  the  emphasis  has  been  on  declarative  knowledge  rather  than  the 
procedural  knowledge  needed  for  effective  thinking — instruction  on  how  to  approach  problems, 
how  to  generate  and  test  hypotheses,  how  to  comprehend  difficult  text,  how  to  make  causal 
inferences.  Yet  we  are  expected  to  instruct  our  own  students  in  the  art  and  the  skill  of  thinking. 
Education  in  most  of  the  disciplines  has  primarily  been  concerned  with  presenting  students  with 
the  "facts”  on  a  variety  of  topics— the  "knowing  that"— while  offering  little  on  how  to  utilize  this 
information  or  how  to  discover  facts  on  their  own — the  "knowing  how." 

Often  faculty  will  tell  me  "Of  course  I  teach  critical  thinking.  What  kind  of  thinking  do 
you  think  I  teach— noncritical  thinking?"  Examples  of  noncritical  thinking  are  the  rote 
applications  of  rules,  verbatim  memory,  or  nondirected  thinking  such  as  day  dreaming.  Let  me 
give  you  an  example  from  psychology,  my  own  area.  It  is  common  in  most  courses  in 
developmental  psychology  to  have  students  learn  Piaget’s  stages  of  cognitive  development  and 
list  them  in  order  by  name  and  age  on  an  exam.  Certainly  this  is  key  information  in  a  class  in 
developmental  psychology,  and  students  need  to  know  it,  but  too  often  the  learning  ends  there — 
students  cannot  understand  any  implications  from  this  work  or  use  it  in  any  meaningful  way 
beyond  repetition.  These  common  practices  are  not  teaching  for  thinking. 
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The  idea  of  critical  thinking  instruction  is  that  the  skills  and  attitudes  of  critical  thinking 
are  deliberately  and  self-consciously  taught.  It  is  the  difference  between  requiring  students  to 
®n§ttge  in  critical  thinking  activities  and  teaching  them  how.  For  example,  a  popular  assignment 
in  many  classes  is  to  have  students  compare  several  events  or  theories.  A  critical  thinking  lesson 
would  explicitly  teach  how  to  make  comparisons.  The  idea  of  having  specialized  classes  in 
critical  thinking  was  not  to  remove  critical  thinking  from  other  classes,  but  to  provide  a  course 
where  instruction  in  how  to  think  critically  is  the  key  focus  with  the  idea  of  using  multiple  types 
of  examples  to  help  with  the  spontaneous  transfer  of  learning,  much  in  the  way  we  have  classes 
that  teach  writing  or  speaking  with  the  goal  of  getting  the  skills  of  good  writing  and  speaking  to 
transfer  to  other  settings. 

The  model  that  I  have  proposed  for  critical  thinking  instruction  consists  of  four  parts: 

A  FOUR-PART  MODEL  FOR  IMPROVING  CRITICAL  THINKING 

1 .  Explicitly  teach  the  skills  of  critical  thinking 

2.  Develop  the  disposition  for  effortful  thinking  and  learning 

3.  Direct  learning  activities  in  ways  that  increase  the  probability  of  transcontextual 
transfer  (structure  training) 

4.  Make  metacognitive  monitoring  explicit  and  overt. 

A  SKILLS  APPROACH  TO  CRITICAL  THINKING 

I  have  proposed  a  taxonomy  of  critical  thinking  skills.  The  taxonomy  is  not  meant  to  be 
an  exhaustive  list— it  is  just  a  starting  point  for  a  serious  discussion  of  what  we  mean  by  critical 
thinking  skills  and  how  we  can  best  define  them  so  that  they  are  meaningful  to  a  broad  audience. 
As  you  can  probably  guess,  I  have  been  working  on  the  development  of  a  test  of  critical  thinking 
based  on  this  taxonomy.  Skill  level  in  each  category  can  be  assessed  separately,  in  addition  to  an 
overall  score.  Other  taxonomies  are  possible  and  a  legitimate  case  can  be  made  for  alternative 
groupings  or  adding  other  skills  and  deleting  some  of  those  that  are  listed.  This  is  not  meant  to 
be  the  definitive  list  of  critical  thinking  skills.  Rather,  it  is  proposed  as  a  concrete  starting  place 
for  the  task  of  deciding  what  we  want  college  graduates  who  are  entering  the  workforce  to  know 
and  be  able  to  do  so  that  they  can  compete  and  cooperate  in  the  world's  market  place  and 
function  as  effective  citizens  in  a  complex  democratic  community. 

Here  are  the  five  category  headings  for  organizing  college-level  critical  thinking  skills: 

1.  Verbal  Reasoning  Skills 

The  skills  listed  under  this  rubric  include  those  that  are  needed  to  comprehend  and 
defend  against  the  persuasive  techniques  that  are  embedded  in  everyday  language 
(also  known  as  natural  language).  Thinking  and  language  are  closely  tied  constructs, 
and  the  skills  included  in  this  section  recognize  the  reciprocal  relationship  between 
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language  and  thought  in  which  an  individual's  thoughts  determine  the  language  used 
to  express  them,  and  the  language  that  is  used  shapes  the  thoughts. 


2.  Argument  Analysis  Skills 

An  argument  is  a  set  of  statements  with  at  least  one  conclusion  and  one  reason  that 
supports  the  conclusion.  In  real  life  settings,  arguments  are  complex  with  reasons 
that  run  counter  to  the  conclusion,  stated  and  unstated  assumptions,  irrelevant 
information,  and  intermediate  steps.  Arguments  are  found  in  commercials,  political 
speeches,  textbooks,  and  anywhere  else  where  reasons  are  presented  in  an  attempt  to 
get  the  reader  or  listener  to  believe  that  the  conclusion  is  true.  The  skills  of 
identifying  conclusions,  rating  the  quality  of  reasons,  and  determining  the  overall 
strength  of  an  argument  should  be  sharpened  in  college  course  work. 

3.  Skills  in  Thinking  as  Hypothesis  Testing 

The  rationale  for  this  category  is  that  much  of  our  day-to-day  thinking  is  like  the 
scientific  method  of  hypothesis  testing.  In  many  of  our  everyday  interactions,  people 
function  like  intuitive  scientists  in  order  to  explain,  predict,  and  control  the  events  in 
their  life.  The  skills  used  in  thinking  as  hypothesis  testing  are  the  same  ones  that  are 
used  in  scientific  reasoning — the  accumulation  of  observations,  formulation  of  beliefs 
or  hypotheses,  and  then  using  the  information  collected  to  decide  if  it  confirms  or 
disconfirms  the  hypotheses. 

4.  Using  Likelihood  and  Uncertainty 

Because  very  few  events  in  life  can  be  known  with  certainty,  the  correct  use  of 
probability  and  likelihood  plays  a  critical  role  in  almost  every  decision.  Huffs  (1954) 
tiny,  popular  book  How  To  Lie  With  Statistics  is  still  widely  quoted  because  it 
explains  how  easy  it  is  to  mislead  someone  who  does  not  understand  basic  concepts 
in  probability.  The  critical  thinking  skills  that  are  subsumed  under  this  heading  are  an 
important  dimension  of  a  college-level  critical  thinking  taxonomy. 

5.  Decision  Making  and  Problem  Solving  Skills 

In  some  sense,  all  of  the  critical  thinking  skills  are  used  to  make  decisions  and  solve 
problems,  but  the  ones  that  are  included  here  involve  the  generation  and  selection  of 
alternatives  and  judging  among  them.  Many  of  these  skills  are  especially  useful  in 
quantitative  reasoning  problems. 

Taken  together  these  five  categories  define  an  organizational  rubric  for  a  skills  approach  to 
critical  thinking.  They  have  face  validity  and  can  be  easily  communicated  to  the  general  public 
and  students  and  offer  one  possible  answer  to  the  question  of  what  college  students  need  to  know 
and  be  able  to  do  to  compete  and  cooperate  in  the  world's  marketplace  and  function  as  effective 
citizens  in  a  democratic  society. 


DISPOSITIONS  FOR  EFFORTFUL  THINKING  AND  LEARNING 

It  is  important  to  separate  the  disposition  or  willingness  to  think  critically  from  the  ability 
to  think  critically.  Some  people  may  have  excellent  critical  thinking  skills  and  may  recognize 
when  they  are  needed,  but  they  may  also  choose  not  to  engage  in  the  effortful  process  of  using 
them.  This  is  the  distinction  between  what  people  can  do  and  what  they  actually  do  in  real  world 
contexts.  Good  instructional  programs  help  learners  decide  when  to  make  the  necessary  mental 
investment  in  critical  thinking  and  when  a  problem  or  argument  is  not  worth  it. 

Dispositions  for  effortful  thinking  and  learning: 

■  Willingness  to  engage  in  and  persist  at  a  complex  task 

■  Conscious  use  of  plans  and  suppression  of  impulsive  activity 

■  Flexibility  or  open-mindedness 

■  Willingness  to  abandon  nonproductive  strategies  in  attempts  to  self-correct 

■  An  awareness  of  social  realities  that  need  to  be  overcome  so  that  thought  can  become 
actions. 

TRANSFER  OF  TRAINING 

In  teaching  for  thinking  the  goal  is  to  have  students  not  only  understand  and  successfully 
use  a  particular  skill  when  it  is  being  taught  but  to  also  be  able  to  recognize  when  that  skill  will 
be  appropriate  in  a  novel  situation.  This  is  the  Achilles'  heel  of  transfer.  The  problem  in  learning 
thinking  skills  that  are  needed  in  multiple  contexts  is  that  there  are  no  obvious  cues  in  the  context 
to  trigger  the  recall  of  the  thinking  skill.  Students  need  to  create  the  recall  cues  from  the 
structural  aspects  of  the  problem  or  argument,  so  that  when  the  structural  aspects  are  present, 
they  can  serve  as  cues  for  retrieval.  A  similar  point  was  made  by  Hummel  and  Holyoak  (1 997) 
who  said,  “First,  thinking  is  structure  sensitive.  Reasoning,  problem  solving,  and  learning... 
depend  on  a  capacity  to  code  and  manipulate  relational  knowledge.” 

When  critical  thinking  skills  are  taught  so  that  they  transfer  appropriately  and 
spontaneously,  students  learn  to  focus  on  the  structure  so  the  underlying  characteristics  become 
salient  instead  of  the  domain  specific  surface  characteristics.  There  is  an  old  saying  in 
psychology  that  “the  head  remembers  what  it  does.”  It  is  important  to  design  learning  activities 
so  that  the  skills  are  encoded  in  a  way  that  they  will  facilitate  their  recall  in  novel  situations  if 

you  want  to  teach  for  transfer.  It  is  what  the  learner  does  that  determines  what  gets  learned,  not 
what  the  teacher  does. 

After  context-rich  information  is  provided,  learners  are  asked  to  perform  certain  tasks  or 
answer  carefully  crafted  questions.  Here  are  some  examples  of  relevant  tasks  and  questions  that 
require  learners  to  attend  to  structural  aspects  of  the  problem  or  argument: 

■  Draw  a  diagram  or  other  graphic  display  that  organizes  the  information 

■  What  additional  information  would  you  want  before  answering  the  question? 
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■  Explain  why  you  selected  a  particular  multiple  choice  alternative.  Which  is  second 
best?  Why? 

■  State  the  problem  in  at  least  two  ways 

■  Which  information  is  most  important?  Which  information  is  least  important?  Why? 

■  Categorize  the  findings  in  a  meaningful  way 

■  List  two  solutions  for  the  problem 

■  What  is  wrong  with  an  assertion  that  was  made  in  the  question? 

■  Present  two  reasons  that  support  the  conclusion  and  two  reasons  that  do  not  support 
the  conclusion 

■  Identify  the  type  of  persuasive  technique  being  used 

■  What  two  actions  would  you  take  to  improve  the  design  of  the  study  that  was 
described? 


METACOGNITIVE  MONITORING 


Metacognition  refers  to  our  knowledge  of  what  we  know  (or  what  we  know  about  what 
we  know)  and  the  use  of  this  knowledge  to  direct  further  learning  activities.  When  engaging  in 
critical  thinking,  students  need  to  monitor  their  thinking  process,  check  whether  progress  is  being 
made  toward  an  appropriate  goal,  ensure  accuracy,  and  make  decisions  about  the  use  of  time  and 
mental  effort.  Students  can  become  better  thinkers  and  learners  by  developing  the  habit  of 
monitoring  their  understanding  and  by  judging  the  quality  of  their  learning.  It  is  the  executive  or 
boss  function  that  guides  how  adults  use  different  learning  strategies  and  make  decisions  about 
the  allocation  of  limited  cognitive  resources.  Numerous  studies  have  found  that  good  learners 
and  thinkers  engage  in  more  metacognitive  activities  than  poor  learners  and  thinkers,  and  that  the 
skills  and  attitudes  of  metacognitive  activities  can  be  taught  and  learned  so  that  students  can 
direct  their  own  learning  strategies  and  make  judgments  about  how  much  effort  to  allocate  to  a 
cognitive  task. 


For  example,  when  students  are  required  to  provide  reasons  and  evidence  to  support  a 
conclusion  and  counterreasons  and  conflicting  evidence,  they  must  focus  on  the  quality  of  the 
thinking  that  went  into  a  decision.  The  giving  of  reasons  forces  thinkers  to  "tickle  their  memory." 
By  this  I  mean  they  have  to  access  information  stored  in  memory.  They  also  have  to  consider 
both  positive  and  negative  evidence.  It  is  well  documented  that  we  tend  to  weigh  evidence  much 

more  heavily  when  it  favors  a  belief  that  we  hold  over  evidence  that  disconfirms  a  personal 
belief. 


There  are  numerous  experimental  demonstrations  of  this  bias  for  confirming  evidence. 
For  example,  in  one  study,  college  students  were  asked  to  write  a  response  to  a  controversial 
question  in  which  they  argued  for  one  side  or  the  other  (Lord,  Ross  &  Lepper,  1979).  Students 
were  then  given  the  results  of  experimental  studies  that  supported  a  "middle  of  the  road"  point  of 
view.  After  reading  the  balanced  review,  subjects  who  favored  the  "pro"  position  of  the 
controversial  question  believed  that  the  objectively  neutral  paper  supported  the  "pro"  position. 
Similarly,  students  who  read  the  "con"  position  and  then  read  the  same  objectively  balanced 
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paper  believed  that  it  favored  the  "con"  position.  Instead  of  bringing  the  two  sides  closer 
together,  as  might  have  been  expected,  the  balanced  paper  drove  them  farther  apart.  Each 
position  focused  on  the  information  that  supported  their  own  point  of  view  and  judged  the 
evidence  that  ran  counter  to  their  favored  position  to  be  weak.  This  same  finding  has  been  found 
with  all  sorts  of  subject  pools,  including  NASA  scientists  who  favor  evidence  and  arguments  that 
support  their  preferred  hypothesis  over  one  that  does  not  (Mynatt,  Doherty,  &  Tweney,  1978). 
Just  telling  people  that  we  tend  to  be  overconfident  and  to  rely  on  confirming  evidence,  so  don't 
do  it,  doesn't  work.  Knowing  that  people  tend  to  be  overconfident  does  not  help  people  to 
become  more  accurate,  nor  does  knowledge  of  the  confirmation  bias  help  people  to  overcome  it. 
Is  it  any  wonder  why  it  is  so  difficult  to  get  people  to  assess  controversial  issues  in  a  fair-minded 
manner?  These  sorts  of  experimental  results  show  that  the  giving  and  assessing  of  reasons  can 
have  beneficial  results  that  improve  the  thinking  process. 

Sometimes,  college  faculty  unwittingly  encourage  the  exact  opposite  of  what  we  want 
students  to  learn.  Consider,  for  example,  many  writing  assignments  where  we  ask  students  to 
take  a  position  on  a  controversial  issue  in  psychology  and  then  to  write  a  position  paper.  This  is 
exactly  the  sort  of  learning  activity  that  would  strengthen  the  bias  to  seek  and  consider  only 
confirming  evidence.  Instead,  we  need  to  help  students  understand  and  be  able  to  explain 
evidence  on  two  or  more  sides  of  an  issue,  such  as  the  evidence  for  and  against  evolutionary 
hypotheses  of  mate  selection  or  for  and  against  the  idea  that  parents  are  important  influences  on 
their  children.  We  want  to  teach  students  to  gather  and  assess  evidence  to  determine  the  best 
conclusion  or  conclusions  and  not  start  with  what  they  believe  is  true  and  consider  only 
supporting  evidence. 

I  hope  that  this  brief  overview  to  the  applications  of  cognitive  science  to  critical  thinking 
instruction  has  convinced  you  that  education  for  thinking  should  be  a  major  objective  of 
education.  When  we  consider  that  many  of  the  today’s  young  adults  will  be  working  at  jobs  that 
do  not  exist  today  and  not  even  science  fiction  writers  can  imagine  and  living  well  into  the 
decades  beginning  with  2060  and  2070,  it  seems  clear  that  the  ability  to  think  critically  and  the 

disposition  to  engage  in  the  effortful  process  of  thinking  are  the  most  critical  components  of  their 
education. 
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A  Framework  for  Critical  Thinking  Research  and  Training 

Susan  C.  Fischer 
Anacapa  Sciences,  Inc. 

My  job  today  is  to  present  to  you  the  results  of  research  we  have  been  conducting  at 
Anacapa  Sciences  over  the  past  nine  months.  The  main  product  of  this  research  has  been  a 
framework  or  model  we  developed  for  critical  thinking  (CT).  We  developed  the  model  with  the 
goal  of  directing  and  guiding  research  on  CT,  particularly  research  investigating  training  issues. 
However,  it  serves  the  additional  purpose  of  providing  a  common  vocabulary  of  CT  that  we  can 
use  throughout  the  workshop  to  discuss  our  ideas. 

Before  I  begin  my  presentation.  I’d  like  to  acknowledge  the  contributions  of  several 
individuals  who  have  greatly  influenced  this  research.  First,  my  colleague  at  Anacapa  Sciences, 
Alan  Spiker,  deserves  equal  credit  and  blame  for  the  ideas  I  will  present  to  you  today.  The 
framework  you  will  hear  about  today  is  the  product  of  such  a  truly  collaborative  effort  that  it’s 
difficult  to  attribute  responsibility  for  any  particular  idea.  Second,  this  work  was  greatly 
enhanced  by  the  help  of  the  officers  of  the  First  Cavalry  Division  at  Fort  Hood,  Texas.  Third, 
this  research  would  not  have  been  possible  without  the  support  we  received  from  the  Army 
Research  Institute.  In  particular,  I’d  like  to  acknowledge  the  significant  contributions  and 
guidance  of  Dr.  Sharon  Riedel. 

I’d  like  to  begin  by  giving  you  a  roadmap  for  my  talk  today.  I  will  briefly  discuss 
historical  conceptions  of  CT  because  I  think  our  framework  is  best  understood  when  placed  in 
front  of  the  backdrop  of  other  researchers’  thoughts.  We  stand  on  the  shoulders  of  those 
researchers  to  perhaps  reach  a  little  farther  in  our  understanding  of  critical  thinking.  I  will  then 
turn  to  a  very  brief  discussion  of  how  we  went  about  developing  our  model  of  CT.  This  will  also 
help  you  to  understand  the  framework  we’ve  developed.  I’ll  then  discuss  the  model  itself  in 
some  detail,  its  contributions  and  limitations,  its  relationship  to  command  and  control,  and  its 
implications  for  training  CT  in  the  military. 

HISTORICAL  CONCEPTIONS  OF  CRITICAL  THINKING 

It  is  clear  that  many  commercial  and  academic  domains  regard  critical  thinking  as  an 
important  characteristic  they  would  like  their  members  to  possess.  For  example,  the 
development  of  CT  skills  in  students  who  attend  public  educational  institutions  has  become  a 
central  component  of  the  United  States’  national  education  goals.  In  the  military,  research  on 
leadership  has  identified  the  need  for  training  programs  that  promote  better  thinking  to  improve 
battle  command  decision  making,  and  several  courses  have  been  developed  and  implemented.  In 
the  public  and  business  sectors,  the  National  League  for  Nursing  has  required  the  demonstration 
of  CT  in  graduates  of  nursing  education  programs,  DeVry  vocational  programs  now  assess  the 
thinking  skills  of  their  students,  and  many  U.S.  corporations  now  provide  employees  with 
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training  in  thinking  skills.  In  short,  increasing  CT  skills  in  U.S.  citizens  has  become  a  significant 
goal  within  government,  public,  and  corporate  arenas. 

Why  has  CT  received  so  much  attention  from  such  a  diverse  group  of  interests?  There 
are,  perhaps,  two  related  answers  to  this  question.  First,  society  is  experiencing  an  increased  need 
for  intellectual  skills  due  to  demands  created  by  developing  technologies.  In  the  past  20  years, 
our  economic  and  social  systems  have  become  dependent  on  complex  technologies,  with 
information  becoming  either  a  primary  or  intermediate  product  that  must  be  processed  to  serve 
decision  making.  One  process  that  works  on  information  is  reasoning,  which  must  be  rational 
and  purposeful  to  be  effective.  When  information  is  incomplete,  uncertain,  or  unreliable,  the 
ability  to  evaluate  its  quality  becomes  paramount  to  competent  decision-making.  Hence,  rational 
CT  is  viewed  as  a  skill  necessary  to  the  manipulation  of  information,  especially  when  the 
information  is  degraded.  Greater  reliance  on  information  will  necessitate  greater  reliance  on  CT 
skills.  All  areas  of  society  are  experiencing  increases  in  available  information  and  a 
corresponding  greater  demand  on  intellectual  processing.  In  some  domains,  however, 
technological  development  has  increased  the  available  information  to  such  a  high  degree  that  job 
demands  may  soon  be  exceeding  current  skill  levels.  Second,  a  number  of  domains  (e.g.,  nursing, 
business,  military  leadership)  have  recognized  the  need  to  improve  CT  of  personnel  whose 
cognitive  skills  they  suspect  are  deficient. 

Feeding  and  supporting  this  increased  interest.  CT  has  also  become  a  recognized 
construct  in  philosophy,  education,  and,  to  a  lesser  degree,  psychology.  If  one  searches  the 
education,  philosophy,  and  psychology  literature  databases  for  the  term  “critical  thinking,”  one 
will  obtain  an  enormous  number  of  citations  in  the  first  two  disciplines,  but  relatively  few  in  the 
third.  Hence,  it  appears  that  the  concept  of  CT  is  not  one  that  has  been  generated  by  psychology, 
but  is  an  issue  that  has  primarily  concerned  philosophers  and  educators.  This  makes  a  lot  of 
sense  because  thinking  is  the  primary  activity  one  engages  in  when  philosophizing  and 
improving  thinking  capabilities  is  a  primary  activity  in  which  educators  engage.  Hence,  the 
construct  of  CT  has  been  developed  by  several  fields,  each  with  its  own  set  of  motivations, 
purposes,  and  biases.  An  unfortunate  fact  is  that  construct  development  has  been  largely  based 
on  philosophical  arguments.  Very  little  empirical  research  has  been  designed  to  develop  the 
theoretical  construct  of  CT.  Compared  to  the  extensive  development  of  the  concept  of 

intelligence,  for  example,  development  of  CT  as  a  theoretical  entity  has  received  almost  no 
empirical  attention. 

Perhaps  because  of  the  different  motivations  of  the  various  fields  who  have  contributed  to 
the  conceptual  development  of  CT,  or  perhaps  because  of  the  lack  of  empirical  grounding  of  the 
concept,  the  present  theoretical  state  of  the  field  is  highly  fragmented.  Although  one  can  extract 
some  general  themes  from  the  literature,  there  is  no  consensus  among  researchers  as  to  the 
concept’s  meaning.  There  appear  to  be  as  many  definitions  of  CT  as  there  are  researchers  who 
study  it.  The  American  Philosophical  Association  attempted  to  provide  some  guidance  to  CT 
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research  by  developing  a  consensus  definition  derived  using  the  Delphi  technique  on  46  critical 
thinking  experts.  However,  the  resulting  lengthy  definition  has  not  adequately  provided  the 
direction  intended  or  required.  Since  1990,  when  the  APA  offered  their  comprehensive 
definition,  authors  who  consider  CT  still  approach  the  topic  using  their  own  definition. 

In  the  absence  of  a  dominant  theory,  one  might  consider  the  three  most  frequently  used 
assessment  instruments  for  evaluating  CT  (the  Watson-Glaser  Critical  Thinking  Appraisal,  the 
California  Critical  Thinking  Skills  Test,  and  the  Cornell  Critical  Thinking  Instrument)  to  be 
defacto  models  of  CT.  Certainly,  they  provide  the  most  commonly  accepted  operational 
definition  of  CT  in  the  research  literature.  However,  studies  have  shown  the  tests  to  have  low 
internal  reliability,  a  lack  of  comparability  among  forms,  and  corresponding  poor  construct 
validity.  Just  as  significant,  it  may  be  that  these  instruments  are  simply  tapping  variability  in 
performance  that  may  just  as  well  be  explained  by  intelligence  and/or  achievement  tests.  For  this 
reason,  we  are  left  with  a  situation  that  produces  great  difficulty  in  communicating  what  we 
mean  when  we  use  the  term  “critical  thinking.” 

Despite  differences  among  conceptions  of  CT,  examination  of  the  literature  reveals  a 
modest  amount  of  overlap  and  redundancy.  Among  these  definitions,  several  “themes”  repeat 
themselves,  of  which  we  identify  six.  For  example,  many,  but  not  all,  theorists  regard  the  ability 
to  use  reasoning  and  informal  applied  logic  as  central  to  CT.  Judgment  or  evaluation  of 
information  or  a  source  is  a  second  common  theme.  Some  theorists  see  judgment  as  the  “critical” 
component  of  CT.  Others  make  no  mention  of  judgment  in  their  definitions  or  discussions.  Other 
themes  seem  to  describe  a  state  of  mind  rather  than  a  skill  or  ability.  For  example,  several 
theorists  describe  CT  as  an  attitude  or  activity  that  is  reflective  or  questioning.  A  few  theorists 
describe  CT  as  a  recursive,  interactive  activity  that  involves  meta-cognition  while  others  simply 
note  that  CT  involves  some  sort  of  mental  process.  Those  who  emphasize  its  meta-cognitive 
nature  view  CT  as  “thinking  about  thinking  that  serves  to  improve  thinking.”  Finally,  several 
definitions  explicitly  emphasize  the  purposeful  nature  of  CT.  For  these  theorists,  CT  is  not  a 
series  of  aimless,  random  thoughts.  It  is  distinguished  from  regular  thinking  in  its  goal-directed 
nature  that  is  applied  to  serve  a  purpose. 

We  must  conclude  that  CT  can  be,  and  has  been,  defined  in  a  number  of  ways.  If  one 
chooses  to  bound  the  definition  narrowly,  then  most  authors  regard  CT  as  the  evaluation  or 
judgment  of  information  and  sources.  If  one  chooses  to  define  CT  broadly,  it  typically  becomes 
all  those  mental  processes  that  are  purposeful.  Or,  as  the  APA  would  suggest,  CT  is  thinking 
that  is  broadly  characterized  by  the  themes  we  have  discussed.  We,  however,  take  a  different 
approach  to  defining  CT,  one  that  embraces  an  empirical  perspective.  Our  emphasis  on 
empirical  evidence  serves  to  de-emphasize  analytic  definitions  of  CT  and  subjects  it  to  scientific 
testing.  We  believe  it  is  time  to  put  away  matters  of  definition  and  move  on  to  the  measurement 
and  testing  of  CT,  for  that  is  where  the  greatest  gains  in  knowledge  and  application  will  be.  In 
summary ,  our  position  is  that  the  utility  of  a  definition  depends  on  its  ability  to  generate  research 
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that  can  lead  to  better  training.  Beyond  these  potential  benefits,  definitions  will  do  little  to 
further  our  understanding  of  CT. 

A  MODEL  OF  CT 

In  fact,  more  than  definitions,  what  is  needed  to  guide  research  and  empirical  testing  is  a 
model  of  CT.  If  we  are  interested  in  improving  thinking,  what  part  of  CT  should  we  focus  on? 
What  part  should  be  trained?  In  short,  while  CT  appears  to  possess  great  promise  in  increasing 
success  in  many  domains,  including  battle  command,  a  model  is  needed  to  guide  training  and 
evaluation.  A  guiding  model  will  generate  research  to  support  training  choices.  It  will  also 
ensure  that  the  training  is  complete,  coherent,  and  consistent. 

To  meet  these  objectives,  we  set  out  to  develop  a  model  of  CT  that  incorporates  existing 
conceptions.  We  sought  to  synthesize  ideas  about  CT  to  develop  a  model  that  was  not  so 
narrowly  defined  as  to  lose  meaning  but  was  also  not  so  broad  as  to  lose  focus.  Criteria  were 
applied  to  ensure  the  model  would  be  useful.  For  example,  we  sought  to  develop  a  model  that 
would  stimulate  the  collection  of  research  data  relevant  not  only  to  developing  the  concept  of 
CT,  but  also  to  guide  the  training  of  CT.  Because  we  were  working  in  the  context  of  battle 
command,  we  wanted  to  develop  a  model  that  would  generate  information  that  could  be  used  to 
populate  military  training  in  CT.  It  was  imperative  that  the  model  could  be  used  to  describe  the 
thinking  tasks  army  officers  use  in  the  context  of  their  work,  specifically,  command  and  control. 

The  model  was  developed  using  a  bottom-up  process  based  on  an  extensive  review  of  the 
educational,  philosophical,  and  psychological  literatures  on  CT.  In  our  review,  we  coded  and 
recorded  any  CT  skill  mentioned  by  an  author.  We  also  noted  any  subject,  situational,  stimulus, 
or  moderating  variable  associated  with  CT.  The  information  gleaned  from  the  literature  review 
provided  critical  input  to  develop  a  CT  model,  moving  forward  a  very  fragmented,  but 
potentially  useful,  construct;  however,  we  shaped  the  model  through  the  use  of  a  top-down 

process  in  which  the  identified  skills  and  variables  were  organized  to  form  a  coherent  picture  of 
CT. 

The  resulting  model  is  closely  tied  to  conceptions  of  CT  offered  by  authors  in  the  fields 
of  philosophy  and  education.  For  example,  it  organizes  a  number  of  variables  that  authors  of  CT 
have  discussed,  such  as  the  influence  of  experience  and  knowledge,  the  relationship  of  CT  to 
other  mental  tasks  (e.g.,  decision  making  and  problem  solving),  and  the  measurement  of  CT. 
However,  it  goes  far  beyond  the  largely  analytic  work  conducted  to  date  by  providing  a 
framework  in  which  CT  can  be  empirically  investigated  as  a  cognitive  process.  Up  to  this  point, 
CT  has  been  treated  as  a  subject  variable  that  can  be  measured  and  correlated  with  other 
variables  of  interest.  We  offer  a  model  that  allows  empirical  investigation  of  CT  as  a 
psychological  construct.  The  most  important  contribution  the  model  provides  is  direction  for 
further  research.  It  does  so  by  generating  testable  hypotheses  regarding  entry  into,  and 
sustainment  of,  a  CT  “state.’'  The  model  also  generates  hypotheses  about  CT’s  measurement, 
detection,  and  products. 
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We  posit  five  essential  elements  to  CT  that  distinguish  it  from  typical  treatments  within 
the  philosophy  and  education  literature,  as  well  as  from  earlier  psychological  treatments.  First, 
we  assume  that  CT  is  intimately  linked  to  a  set  of  initiating  situational  or  contextual  conditions 
that  increase  the  likelihood  that  an  individual  will  use  his/her  CT  skills  and  thus  enter  the  CT 
“state.”  In  this  way,  we  have  intentionally  distanced  ourselves  from  a  large  segment  of  the 
educational  and  philosophical  literature,  which  primarily  treats  CT  as  an  individual  difference 
measure.  Our  focus  is  on  identifying  the  processes  that  are  brought  to  action  in  response  to 
definable  initiating  conditions,  and  our  objective  is  a  compact  yet  powerful  theory  of  the  skills 
required  to  process  information  deeply  in  response  to  measurable  initiating  conditions.  Research 
on  this  theoretical  tenet  may  serve  to  generate  a  priori  predictions  of  when  CT  will  occur. 

Second,  we  treat  CT  as  a  highly  stimulus-bound  phenomenon  in  which  deep  cognitive 
processing  is  imposed  on  a  circumscribed  stimulus-information  complex  to  assess  its  relevance, 
accuracy,  completeness,  clarity,  logical  consistency,  and/or  precision.  By  bounded,  we  mean  that 
the  to-be-processed  stimulus  can  be  physically  isolated  and  specified  in  advance.  This  tenet 
affords  investigation  of  external  stimulus  factors  that  govern  entry  into  or  maintenance  of  the  CT 
“state.” 

Third,  we  consider  CT  to  be  truly  time-limited,  wherein  an  individual  may  execute  the 
necessary  skilled  processes  for  only  a  few  minutes  (or  less)  before  he/she  must  “come  out”  and 
assess  the  status  of  what  has  been  accomplished.  A  likely  reality  is  that  CT  processes  are 
continually  being  interrupted  in  order  to  consult  with  others,  view  the  consequences  in  the 
external  world,  or  simply  “take  a  break”  from  the  mental  effort.  A  ripe  area  for  study  will  be  to 
identify  the  stimulus  and  contextual  conditions  that  precipitate  these  breaks  and  to  develop 
countermeasures  that  serve  to  keep  people  longer  in  a  CT  state. 

Fourth,  we  assume  that  engaging  in  CT  processes  has  state-like  consequences  in  which 
the  individuals  experience  emotions,  motivations,  and  other  phenomenological  experiences  that 
are  reportable.  While  the  literature  has  made  passing  reference  to  CT  as  being  “effortful”  or 
work,  we  propose  that  state  properties  are  a  vital  aspect  of  CT  as  a  phenomenon.  In  particular, 
negative  experiential  consequences  of  the  state  that  seem  to  accompany  CT  would  offer  a 
plausible  explanation  as  to  why  people  do  not  engage  in  it  more  often.  Besides  the  negative 
affect  generated  during  CT,  there  may  be  other  state-like  properties  of  CT  that  might  have 
implications  for  research.  For  example,  it  may  be  possible  to  correlate  psychophysiological 
measures  with  CT  occurrence. 

Finally,  in  view  of  the  previous  four  tenets  described  above,  we  believe  CT  can  be 
subjected  to  experimental  manipulation  given  proper  controls.  This  is,  perhaps,  the  most 
important  outgrowth  of  our  assumptions,  as  it  implies  one  can  manipulate  the  likelihood  of  CT 
through  systematic  changes  in  the  stimulus  and/or  contextual  conditions.  This  view  stands  in 
sharp  contrast  to  the  bulk  of  the  literature  that  treats  CT  as  a  response  measure  and  an  indicator 
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of  individual  differences  in  which  people’s  scores  on  some  standardized  test  of  CT  are  correlated 
with  other  subject  characteristics. 

Despite  our  caveats  about  definitions,  we  offer  one  for  CT  that  summarizes  our 
theoretical  tenets.  However,  our  definition  should  not  be  taken  too  seriously,  for  we  believe  the 
most  fruitful  definition  will  be  operational  and  developed  in  the  context  of  empirical  study. 
Critical  thinking  is  a  time-limited  mode  of  deep  processing  that  is  goal-directed,  stimulus-driven, 
and  context-bound.  Its  constituent  processes  are  tightly  wound  and  the  experiential 
consequences  of  being  in  the  state  are  generally  unpleasant. 

The  opportunities  for  a  CT  episode  are  set  in  motion  when  a  set  of  situational  conditions 
arise  that  demand  or  at  least  make  it  highly  desirable — that  the  subject  engage  in  deep 
processing  to  achieve  a  desired  analysis  of  the  stimulus  information.  Situational  conditions  are 
environmental  factors  that  may  include  variables  pertaining  to  the  stimulus,  task,  and 
surrounding  context.  Situational  conditions  do  not  include  subject  variables,  however.  As  shown 
in  the  model  of  CT  depicted  in  Figure  1,  two  types  of  situational  conditions  are  distinguished, 
defining  and  predictive.  Defining  situational  conditions  are  ones  that  are  necessary  for  entry  into 
a  CT  episode.  We  posit  that  three  must  be  present  in  the  situation  for  entry  into  and  maintenance 
of  CT.  These  include  (1)  substantive  information  exists  in  the  stimulus  (there  is  information  that 
requires  processing);  (2)  the  required  activities  cannot  be  executed  by  algorithmic  methods,  i.e., 
they  are  not  procedural  or  routine;  and  (3)  time  is  not  severely  limited,  i.e.,  sufficient  time  is 
available  to  engage  in  CT.  A  set  o f  predictive  situational  conditions  is  also  posited.  These  are 
conditions  that  increase  the  likelihood  an  individual  will  enter  into  a  CT  episode.  For  example, 
situations  where  there  is  no  single  answer  to  a  problem,  or  in  which  conflicting  information  is 
received,  would  tend  to  elicit  a  CT  episode. 

Besides  the  situational  conditions,  predisposing  attitudes  are  posited  that  make  the  subject 
more  or  less  likely  to  engage  in  CT  processing.  Predispositions  are  essentially  subject  variables 
that  represent  individual  differences,  whetherybreJ  or  modifiable,  within  the  population  of 
concern  that  are  related  to  CT  in  a  specific  way.  They  influence  the  likelihood  of  a  person 
entering  into,  or  maintaining,  a  state  of  CT,  and  like  situational  conditions,  they  serve  as  input 
conditions  for  the  use  of  CT  skills  (see  Figure  1).  They  do  not  include  predispositions  that 
govern  how  well  someone  thinks  critically,  however. 
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Figure  1:  Depiction  of  CT  Model 

The  model  also  incorporates  a  set  of  moderating  variables  representing  individual 
difference  factors  that,  while  not  directly  or  causally  related  to  one’s  use  of  CT  skills,  might 
mitigate  the  effects  of  CT  skills  through  interactions  with  other  variables.  Moderating  variables 
are  not  part  of  the  stimulus  situation,  but  tend  to  reflect  subject  characteristics  (e.g.,  domain 
expertise,  recent  experience,  education)  that  would  need  to  be  considered  when  designing  test 
procedures  or  forming  subject  groups.  For  example,  highly  experienced  individuals  may  have  a 
greater  tendency  to  enter  into  and  maintain  CT,  or  they  may  have  the  requisite  knowledge  to  use 
certain  CT  skills. 

Figure  1  also  shows  four  meta-tasks  that  serve  to  delineate  the  task  requirements  and 
general  purpose  of  a  CT  episode.  They  provide  a  higher  order  structure  for  specific  tasks,  which 
themselves  are  part  of  the  theory.  In  a  strict  sense,  the  meta-task  requirements  of  the  episode 
form  part  of  the  stimulus  context  and  thus  dictate  the  specific  response  that  will  be  required  to 
successfully  end  a  given  CT  “episode.”  CT  skills  serve  the  purpose  of  completing  tasks  and 
meta-tasks,  hence,  in  Figure  1  the  relationship  between  these  two  theoretical  constructs  is 
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depicted  as  a  directional  one  where  the  performance  of  CT  skills  serves  as  input  to  tasks  and 
meta-tasks. 


As  noted  above,  an  interesting  corollary  to  the  present  approach  is  the  postulation  that 
individuals  who  engage  in  CT  for  any  substantive  length  of  time  experience  affective  reactions 
that  are,  by  and  large,  negative.  Consistent  with  the  CT  literature,  we  hypothesize  that  indi¬ 
viduals  who  maintain  CT  do  so  in  part  because  they  are  able  to  maintain  a  neutral  state.  In  other 
words,  we  do  not  suggest  there  are  positive  intrinsic  rewards  associated  with  the  process  of  CT, 
although  there  are  probably  positive  outcomes  that  result  from  the  application  of  CT.  Several 
affective  reactions  are  specified,  the  predominant  byproduct  being  one  of  mental  fatigue  or 
workload.  In  Figure  1,  experiential  consequences  are  depicted  as  a  byproduct  of  performing  CT 
skills,  which  are  shown  within  the  central  region  labeled  “Critical  Thinking  State.” 


The  inner  part  of  the  figure  depicts  blocks  of  CT  skills  that  offer  a  scheme  for  organizing 
the  myriad  skills  and  processes  that  are  involved  when  an  individual  engages  in  CT  and  a  model 
of  the  types  of  processes  that  one  engages  in  when  entering  CT.  Once  in  the  state,  an  individual 
will  initiate  CT  processes  that  can  be  categorized  into  one  of  three  stages.  As  shown  in  Figure  I, 
these  correspond  to  interpretation  of  the  stimulus  materials,  reasoning  with  some  facet  of  the 
extracted  information,  and  then  engaging  in  some  type  of  meta-cognitive  monitoring  that  links 
the  CT  process  with  other  CT  episodes,  other  aspects  of  the  environment,  or  the  meta-task 
demands.  Comparison  of  outcomes  to  a  set  of  stop  criteria  will  lead  to  initiation  of  the  required 

decision,  a  judgment,  understanding  of  the  material,  a  solution,  or,  perhaps,  another  cycle  of  CT 
processing. 


The  separation  of  CT  processes  and  skills  into  the  three  stages  is  a  preliminary  attempt  to 
model  CT  as  a  set  of  cognitive  processes.  However,  strict  unambiguous  separation  among  skills 
is  difficult  at  this  point,  and  it  is  unclear  whether  this  particular  partitioning  of  skills  is  the  most 
useful.  Tables  1,2,  and  3  offer  a  complete  list  of  skills  incorporated  into  the  model. 
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Table  3 


CRITICAL  THINKING  IN  COMMAND  AND  CONTROL 

The  real  test  for  any  model  is  the  development  and  evaluation  of  models  that  instantiate 
the  model  s  elements  in  real-world  applications.  Hence,  an  important  component  of  our  research 
has  been  to  determine  if  a  model  of  CT  could  be  developed  for  the  domain  of  Army  command 
and  control  (C2).  We  attempted  to  validate  the  content  of  the  model  by  conducting  a  combined 
survey  and  after-action  interview  with  Army  subject  matter  experts  at  Ft.  Hood,  Texas.  The 
focus  of  the  survey  was  to  ascertain  whether  the  model  adequately  delineated  CT  skills, 
exemplified  them  in  terms  understandable  to  Army  officers,  and  represented  their  relationship  to 
battle  command  tasks  in  meaningful  ways.  A  survey  instrument  was  developed  and  administered 
to  address  these  questions.  After-action  interviews  were  also  conducted  to  obtain  a  better 
understanding  of  particular  survey  responses. 

The  results  of  the  survey  and  interviews  are  represented  in  Figure  2.  Officers  noted  that 
the  predisposing  attitudes  of  being  persistent,  confident,  withholding  judgment,  being  reasonable 
and  willing  to  expend  effort  were  critical  to  C2.  They  also  confirmed  that  defining  situational 
conditions  necessary  for  CT  were  that  the  task  be  non-algorithmic,  there  must  be  sufficient  time 
for  thinking  to  occur,  and  there  must  be  sufficient  content  in  the  task.  The  predictive  situational 
conditions  they  identified  were  that  the  situation  presented  conflicting  information  and  that  the 
stakes  were  high  and  therefore  motivating.  The  officers  also  noted  that  the  resolution  of  the 
situation  could  be  achieved  with  more  than  one  solution.  The  key  predisposing  attitudes,  defining 
situational  characteristics,  and  battle  C2  tasks  identified  by  the  survey  respondents  were 
incorporated  into  the  model. 
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FIGURE  2:  Depiction  of  CT  Model  Instantiated  in  C2  Tasks 

Survey  respondents  indicated  a  set  of  C2  tasks  for  each  set  of  CT  skills.  For  example, 
mission  analysis,  development  of  commander’s  intent,  interpreting  the  commander’s  intent, 
battle  tracking,  and  intelligence  analysis  were  all  thought  to  heavily  tap  interpretation  skills. 
Figure  2  shows  the  C2  tasks  associated  with  reasoning  and  meta-cognitive  skills  as  well.  The 
results  indicated  that  meta-task  requirements  for  CT  include  only  decision  making  and  solution 
(planning).  The  strong  need  in  battle  command  to  make  decisions  and  execute  actions  rules  out 
tasks  in  which  the  sole  purpose  is  to  simply  understand  or  to  make  a  judgment  of  quality.  Figure 
2  also  lists  some  example  outcome  measures  that  could  be  used  to  assess  the  ultimate  products 
created  from  variation  in  CT  performance. 

IMPLICATIONS  FOR  TRAINING 

The  model  we  have  developed  to  explain  CT  has  specific  implications  regarding  its 
training.  For  example,  if  CT  is  stimulus  driven,  then  training  should  include  multiple  scenarios 
that  are  information-rich.  The  scenarios  should  promote  the  necessary  CT  skills  and  advanced 
training  principles,  such  as  deliberate  practice,  should  be  used  to  structure  education.  If  CT  is 
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stimulus  driven,  research  is  necessary  to  identify  the  content  parameters  of  the  stimuli  that  are 
most  effective  at  eliciting  CT  episodes.  The  goal  of  training  should  be  to  maximize  transfer  of 
CT  skills  to  the  unpredictable  battlefield  environment.  For  this  to  happen,  the  stimulus 

dimensions  of  the  battlefield  related  to  CT  must  be  identified  and  utilized  within  a  training 
environment. 

If  CT  is  multidimensional,  as  indicated  by  the  set  of  CT  skills  provided  in  the  model,  a 
variety  of  training  formats  will  be  required.  Each  dimension  of  CT  is  likely  to  require  a  different 
approach.  The  Web  may  provide  the  flexibility  necessary  to  house  varied  training  formats.  It 
can  also  provide  the  opportunity  for  self-paced  practice,  which  is  likely  to  be  necessary  because 
of  CT  s  multidimensional  nature.  This  aspect  of  CT  may  also  require  that  instructors  take  the 
role  of  a  coach  or  tutor,  providing  guidance  as  necessary  and  monitoring  progress.  Finally,  team 
training  may  be  needed  for  skills  that  emerge  in  the  context  of  a  group. 

If  CT  is  fundamentally  cognitive,  a  cognitive  task  analysis  will  be  necessary  to  guide 
instructional  development.  In  fact,  multiple  cognitive  task  analysis  paradigms  will  be  necessary 
to  capture  the  elemental  cognitive  functions  that  determine  CT. 

If  CT  is  a  learned  skill,  curricula  will  vary  based  on  rank.  Training  must  also  consider 
the  student  s  learning  curve.  Scenarios  will  grow  in  complexity,  as  will  performance 
expectations  and  evaluation  standards.  Research  is  needed  to  determine  how  CT  increases  with 
experience.  Research  is  also  needed  to  develop  innovative  training  techniques  because  there  is 
little  extant  empirical  research. 

CONTRIBUTIONS  AND  LIMITATIONS  OF  THE  MODEL 

To  conclude  my  talk  today.  I’d  like  to  address  the  contributions  our  model  makes  to 
current  thinking  about  CT,  as  well  as  its  limitations.  We  believe  the  most  important  benefit  the 
model  offers  is  that  it  affords  and  lays  the  groundwork  for  empirical  testing  of  CT.  Today, 
we  ve  discussed  just  a  few  of  the  testable  predictions  generated  by  the  model.  Hypotheses 
concerning  predispositions,  defining  and  predictive  situational  factors,  stimulus  dimensions, 
moderating  variables  such  as  expertise,  meta-tasks  such  as  decision  making,  and  particular  CT 
skills  can  all  be  evaluated  in  the  context  of  a  guiding  and  organizing  framework.  We  encourage 
other  researchers  to  also  evaluate  the  core  tenets  of  the  model.  Further  empirical  research  guided 
by  this  nascent  model  will  serve  to  increase  our  knowledge  of  CT  and  develop  better  theories 
grounded  in  real  world  phenomena. 

A  second  major  contribution  of  the  model,  if  adopted  by  the  research  community,  is  that 
it  provides  a  common  vocabulaiy  for  discussing  CT.  Because  the  field  is  so  fragmented  and 
because  most  studies  do  not  provide  empirical  data  that  would  incorporate  operational 
definitions,  it  is  very  difficult  to  determine  if  authors  are  really  talking  about  the  same  construct 
when  they  speak  of  CT.  Hence,  we  hope  the  model  will  help  to  synthesize  and  organize  multiple 
perspectives,  thereby  incorporating  and  embracing  highly  diverse  approaches  to  defining  CT. 
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I  now  turn  to  some  of  the  model’s  limitations.  Some  may  see  the  boundaries  set  on  CT 
by  the  model  as  exclusive  of  important  conceptions  or  skills.  Although  the  model  uses  a  broad 
definition  of  CT  that  is  very  inclusive,  it  may  exclude  some  notions.  For  this  reason,  some  may 
be  dissatisfied  with  our  approach.  On  the  other  hand,  others  who  wish  to  limit  CT  to  critical 
evaluation  and  judgment  may  regard  our  rather  inclusive  model  as  too  broadly  specified.  We 
believe  that,  ultimately,  empirical  phenomena  will  serve  to  identify  the  most  appropriate  and 
useful  boundaries  for  a  model  of  CT. 

Despite  our  stated  empirical  approach  to  develop  CT  as  a  psychological  construct,  it  is 
somewhat  ironic  that  the  model  is  currently  untested.  We  see  this  as  the  model’s  most  important, 
although  hopefully  temporary,  limitation. 

ADDENDUM:  ADDITIONAL  RESEARCH  ISSUES 

Following  our  presentation,  workshop  attendees  were  encouraged  to  identify  additional 
research  issues  that  might  be  investigated  regarding  CT  and  its  development  within  the  military. 
What  follows  is  a  brief  summary  of  the  recommendations  provided  by  the  workshop  participants. 

In  general,  workshop  participants  were  interested  in  teaching  methods  for  increasing  CT 
within  teams.  Many  tasks  carried  out  by  Army  officers  occur  in  a  team  context  where  resources, 
information,  and  task  products  are  shared  and  interdependent.  Therefore,  it  makes  sense  that  the 
integrated  workings,  roles,  and  interrelationships  of  the  commander  and  his  staff  may  affect 
elements  of  CT.  There  are  currently  no  known  studies  of  CT  within  a  team  context;  hence,  this 
is  a  ripe  area  for  future  research. 

Perhaps  the  most  needed  research  concerning  CT  is  its  measurement.  As  previously 
noted,  extant  instruments  are  limited  to  discriminating  individual  differences  based  on  only  a  few 
of  the  many  skill  dimensions  CT  supposedly  incorporates.  Workshop  participants  noted  that  CT 
measurement  is  a  very  complex  endeavor  in  part  because  it  is  difficult  to  identify  true  CT  as 
distinct  from  after-the-fact  rationalization.  Any  verbal  justification  of  a  decision,  for  example, 
could  reflect  the  actual  thinking  that  an  individual  used  to  complete  the  task,  or  it  could  simply 
be  a  fabricated,  satisfying  “story”  that  fulfills  the  need  to  explain  the  decision.  The  challenge  for 
future  research  in  the  measurement  of  CT  will  be  to  reliably  detect  true  CT  and  distinguish  it 
from  rationalization. 

Making  the  measurement  situation  even  more  problematic  is  the  demonstrated  lack  of 
reliability  of  the  existing  instruments’  scales  and  forms.  Therefore,  better  measurement 
techniques  are  needed  to  capture  CT  as  a  mental  process  and  “state”  as  proposed  by  the  model, 
and  to  evaluate  the  quality  of  CT  reliably. 

Issues  regarding  the  training  of  CT  should  also  be  investigated  in  future  research. 

Perhaps  the  key  issue  that  has  interested  educational  researchers  for  many  years  now  is  still  the 
most  important  training  issue:  Do  increases  in  CT  obtained  from  training  in  one  domain  transfer 
to  other  domains?  Workshop  participants  asked  the  following  related  question:  Is  it  better  to 
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infuse  the  training  of  CT  throughout  the  Army's  curriculum  (or  at  least  in  certain  training,  such 
as  the  Command  General  Staff  Officer  Course)  or  should  it  be  taught  as  a  separate  course? 
Should  CT  be  taught  in  the  abstract,  in  hope  of  increasing  the  likelihood  of  transfer,  educating 
students  in  the  principles  of  argument  evaluation,  and  the  identification  of  assumptions  for 
example,  or,  should  CT  be  taught  in  the  context  of  a  combat  domain?  Should  CT  be  taught 
overtly  such  that  students  know  they  are  being  instructed  in  methods  that  would  ostensibly 
improve  their  thinking,  or,  should  training  that  would  increase  thinking  be  delivered  to  students, 
but  veiled  in  the  guise  of  material  of  the  content  domain?  It  is  possible  that  awareness  of 
thinking  is  actually  detrimental  to  thinking;  so,  does  awareness  of  CT  skills  improve 
performance  or  will  it  backslide?  Instructional  developers  find  these  issues  to  be  continual 
problems  for  the  design  of  their  courses. 

A  final  set  of  research  questions  concerned  the  implementation  of  CT  within  the  Army. 
Some  workshop  participants  questioned  whether  CT  is  even  useful  or  valid  for  the  military.  On 
the  other  hand,  some  questioned  whether  the  Military  Decision  Making  Process  (MDMP)  is  still 
appropriate  for  an  Army  that  is  digitized,  and  facing  more  and  more  ill-specified  situations  that 
require  adaptation  and  novel  solutions.  CT’s  role  in  the  context  of  these  changes  the  army  is 
currently  facing  is  another  issue  to  be  considered  in  future  research. 

ADDENDUM:  SUGGESTED  CHANGES  TO  MODEL 

In  discussion,  workshop  participants  recommended  several  changes  to  the  model.  What 
follows  is  our  interpretation  and  judgment  of  those  recommendations. 

■  Recommendation  #1:  Change  the  model  from  static  to  dynamic.  It  is  not 

completely  clear  what  exactly  is  meant  by  “dynamic”  in  this  recommendation. 
However,  mention  was  made  of  the  need  for  a  representation  of  a  sequence  of  events 
in  the  model.  It  is  my  belief,  based  on  inferences,  that  workshop  participants  see  the 
need  for  a  process  model  of  CT.  In  other  words,  they  would  like  a  CT  model  to 
describe  the  process  of  CT. 


In  our  view,  the  component  of  the  model  that  needs  to  be  made  dynamic  are  the  CT 
skills,  depicted  in  Figure  1  as  a  “state.”  A  psychological  state  should  not  be 
represented  by  skills  because  skills  are  merely  one  of  the  observable,  and  hopefully 
measurable,  outcomes  of  a  state.  Hence,  they  cannot  represent  the  state  itself,  only  its 
byproducts.  We  know  people  have  skills  when  they  demonstrate  they  can  do’ 
something  and  when  those  actions  are  influenced  by  training.  Hence,  inclusion  of 
skills  that  represent  the  outcome  of  a  state  is  a  reasonable  component  to  include  in  a 
nascent  model.  Skills  are  not  the  same  as  abilities,  which  are  theoretically  individual 
difference  variables  that  are  typically  hypothesized  to  be  heavily  influenced  by 
heredity.  Logically,  what  is  needed  to  describe  the  state  is  the  engine  that  drives  CT. 

Unfortunately,  none  of  the  “engines”  that  were  offered  at  the  workshop  as  possible 
driving  forces  for  CT  seem  to  fit  our  conception.  For  example,  naturalistic  decision 
making  or  recognition-primed  decision  making  (RPDM)  is  a  nonanalytic  process  that 
we  believe  runs  parallel  to  CT.  Even  if  one  provides  a  meta-cognitive  overlay  on 
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RPDM,  the  basic  element  is  recognitional  and  nonanalytic,  is  performed  rapidly,  and 
may  well  be  algorithmic.  The  latter  feature  runs  counter  to  the  situational  elements 
that  we  propose  initiate  CT  episodes.  At  this  point,  the  cognitive  elements  that  drive 
CT  have  not  been  identified. 

■  Recommendation  #2.  The  model  should  include  a  condition  that  initiates  a  CT 
episode.  As  was  true  for  Recommendation  #1,  we  basically  agree  with  this  second 
bit  of  advice.  We  are  currently  considering  a  number  of  theoretical  conceptions  that 
would  serve  to  initiate  CT.  For  example,  a  CT  episode  may  be  entirely  started  by  the 
situation  and/or  stimulus.  However,  there  must  also  be  some  cognitive  element  that 
recognizes  that  situation  or  stimulus,  interprets  it,  and  at  the  very  least,  sends  a 
message  to  start  the  CT  process.  The  question  becomes  whether  initiation  of  CT  is 
always  a  conscious  decision  point.  It  seems  that  there  are  times  when  people  make  a 
conscious  decision  to  engage  in  deep  thinking.  Most  of  the  time,  however,  the 
phenomenological  experience  is  one  of  being  “pulled”  into  the  state  by  some  external 
stimulus.  It  may  be  possible  to  solve  this  issue  by  inserting  a  model  element  between 
the  situation  and  the  state  that  signifies  a  perception  of  the  situation. 

■  Recommendation  #3:  Connect  the  unpleasant  consequences  element  to  the  stop 
criteria.  In  other  words,  when  the  consequences  get  too  unpleasant,  perhaps  after 
reaching  some  threshold,  people  tend  to  disengage  CT.  A  dynamic  process  model  of 
the  CT  state  may  have  implications  for  how  the  negative  affective  consequences  work 
with  the  stop  criteria.  Other  than  acknowledging  the  relationship  between  the 
negative  consequences  and  maintenance  of  CT,  clarification  of  the  relationship  will 
have  to  wait  for  further  development  of  the  process  model. 

A  related  recommendation  was  that  some  workshop  participants  disagreed  that  CT 
has  negative  experiential  consequences.  Individuals  reported  enjoying  CT  and  could 
maintain  the  state  for  long  periods  of  time.  Hence,  there  may  well  be  differences 
among  individuals  in  their  affective  experiences  of  CT.  Also,  some  participants 
wanted  to  change  the  predisposing  attitude  of  “resistant  to  anxiety”  to  “tolerance  for 
ambiguity.”  This  recommendation  was  made  because  some  individuals  reported  that 
they  never  experience  anxiety  when  they  are  in  a  CT  state.  Hence,  they  preferred  to 
express  their  experience  as  a  tolerance  for  ambiguity. 


■  Recommendation  #4.  Include  inhibiting  mechanisms  that  are  culturally  based. 

A  lengthy  discussion  at  the  workshop  centered  on  how  the  military  discourages  CT 
because  of  the  culture  of  action  and  decision  making.  A  related  issue  was  the  need  to 
carefully  express  oneself  when  you’re  critically  thinking  so  as  not  to  offend  others  by 
appearing  to  be  too  critical,  questioning,  or  mistrustful.  The  ramifications  of 
employing  CT  in  the  context  of  social  interactions  was  not  considered  in  our  model 
development.  However,  it  may  well  be  possible  to  capture  some  of  the  effects  of 
cultural  and  social  situations  by  testing  them  as  moderating  variables  in  the  model. 
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A  Three-part  Theory  of  Critical  Thinking: 
Dialogue,  Mental  Models,  and  Reliability 

Marvin  S.  Cohen,  Ph.D. 

Cognitive  Technologies,  Inc. 

SHOULD  THE  ARMY  BE  INTERESTED  IN  CRITICAL  THINKING1? 

Is  critical  thinking  important?  And  if  so,  why?  A  small  set  of  themes  appears  over  and 
over  in  the  prefaces  and  introductions  of  the  dozens  of  critical  thinking  textbooks  that  are  in 
print.  Claims  fall  into  three  groups:  problem  difficulty,  including  increasing  complexity  of 
problems,  changing  nature  of  problems,  and  information  overload;  decentralized  social  and 
organizational  structure,  including  increasing  responsibility  and  need  for  initiative,  increasing 
participation  in  teams  with  diverse  membership,  and  increasing  need  for  independent  thinking; 
high  stakes,  including  important  public  policy  issues  and  personal  decisions  in  an  increasingly 
competitive  career  environment. 

Do  conditions  for  the  use  of  critical  thinking  apply  in  the  Army?  The  answer  certainly 
appears  to  be  yes.  There  is  a  growing  interest  in  critical  thinking  among  Army  instructors  and 
researchers,  which  seems  warranted  by  the  complexity  and  changing  character  of  military 
planning  and  operations;  decentralization  of  the  organizational  structure  (e.g.,  the  demands  of 
leadership,  coordination,  and  initiative  within  every  echelon);  and  high  stakes  personally, 
organizationally,  and  for  the  nation  as  a  whole.  In  addition,  the  direction  of  change  in  the  Army 
promises  to  make  critical  thinking  even  more  important.  These  changes  include  the  growing 
complexity  of  military  tasks,  the  rapid  evolution  of  technology  and  missions,  the  flood  of 
information  unleashed  by  the  new  technology,  increasing  diversity  of  military  organizations,  and 
the  growing  interest  in  tactics  that  rely  on  initiative  by  local  commanders. 

A  good  case  can  be  made  that  critical  thinking  is  an  important  Army  battlefield  skill,  and 
that  its  importance  is  likely  to  increase.  But  it  is  important  to  get  beyond  the  rhetorical 
compatibility  of  claims  for  critical  thinking  and  Army  needs— and  to  evaluate  the  prospects  of  a 
match  at  a  deeper  and  more  detailed  level.  To  dramatize  the  need  for  clarification  and  coherence, 
let  us  play  devil  s  advocate.  The  current  state  of  critical  thinking  research  and  instruction  leaves 
unanswered  some  important  questions  about  the  application  of  critical  thinking  to  the  Army 
tactical  battlefield  domain: 


1  This  research  was  funded  by  Contract  No.  DASW01-00-C-3010  with  the  Army  Research  Institute,  Fort 
Leavenworth  Field  Unit.  Thanks  to  Dr.  Sharon  Riedel  for  her  help  throughout  this  project. 
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1.  Is  critical  thinking  consistent  with  tactical  battlefield  constraints? 

■  Will  critical  thinking  on  the  battlefield  take  too  much  time?  Would  that  time  be  put  to 
better  use  gaining  a  jump  on  the  enemy? 

■  Will  critical  thinking  result  in  a  loss  of  the  confidence  necessary  for  decisive 
leadership  and  action?  Will  it  undermine  the  “will  to  fight”? 

2.  Is  critical  thinking  consistent  with  other  battlefield  skills? 

■  Will  critical  thinking  skills  trump  experience  or  leadership  qualities  on  the  battlefield, 
which  might  in  fact  lead  to  better  decisions? 

■  Will  critical  thinking  be  too  “critical”?  Will  it  stifle  innovation  or  the  development  of 
new  tactics  and  techniques? 

3.  Is  critical  thinking  appropriate  for  military  organizational  structure? 

■  Will  critical  thinking  encourage  inappropriate  initiative?  Will  it  disrupt  the  chain  of 
command  and  degrade  coordination  and  synchronization  on  the  battlefield?  Put 
another  way,  is  the  Army  too  centralized  and  hierarchical  for  critical  thinking  to 
flourish? 

■  Will  critical  thinking  hinder  the  development  of  trust  within  diverse,  multi-national 
operations  because  it  is  “Western,  masculine,  individualistic,  adversarial,  and  coldly 
rational”  (Atkinson,  1998:  p.121). 

4.  Will  critical  thinking  fit  into  Army  training? 

■  Are  there  “right  answers”  in  critical  thinking?  If  so,  isn’t  this  just  a  new  phrase  for 
teaching  doctrine  and  tactics,  which  we  already  do?  If  not,  what  good  are  skills  that 
can’t  be  evaluated?  How  can  we  know  they  will  improve  performance? 

■  Will  critical  thinking  instruction  consume  too  much  training  time?  How  will  we 
persuade  instructors  to  provide  that  time?  Does  critical  thinking  require  technical 
training  in  logic  or  decision  theory?  Does  it  require  stand-alone  courses?  How  will  we 
persuade  students  to  devote  their  time  to  the  study  of  critical  thinking? 

This  article  can  only  scratch  the  surface  in  trying  to  respond  to  these  challenges.  It  is  a 
very  brief  abridgment  of  Cohen,  Salas,  and  Riedel  (2001),  which  provides  more  depth  and  detail, 
but  is  still  only  a  start.  The  research  had  two  main  goals:  first,  to  draw  a  map  that  links  disparate 
regions  of  the  critical  thinking  field,  and  second,  to  use  the  map  to  navigate  toward  a  more 
insightful  theory  of  critical  thinking,  which  will  support  the  development  of  more  effective 
methods  for  improving  it  in  Army  battlefield  command  teams. 

THREE  COMPONENTS  OF  CRITICAL  THINKING 

The  essence  of  our  theory  is  that  critical  thinking  skill  is  exemplified  by  asking  questions 
about  alternative  possibilities  in  order  to  achieve  some  objective.  Asking  and  answering 
questions  is  a  skill  of  dialogue.  Alternative  possibilities  are  represented  by  mental  models.  A 
process  of  questioning  mental  models  is  adopted  because  of  its  reliability  for  achieving  the 
purposes  of  the  participants  within  the  available  time.  Thus,  the  theory  of  critical  thinking  draws 
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on  and  synthesizes  research  on  three  separate  topics:  cognitive  theories  of  reasoning  according  to 
which  alternative  possible  situations  are  represented  by  mental  models;  normative  models  of 
critical  discussion  in  which  a  proponent  must  defend  a  claim  against  an  opponent  or  critic;  and 
models  of  cognitive  mechanisms  and  of  the  environment  which  enable  us  to  assess  the  reliability 
of  the  processes  by  means  of  which  we  form  beliefs  and  make  choices. 

Critical  thinking,  like  an  onion,  has  a  multi-layered  structure  (Figure  1).  Each  of  the  three 
layers  is  associated  with  distinctive  criteria  of  performance,  which  progress  from  internal  to 
external  in  their  focus: 

1 .  At  its  innermost  core  critical  thinking  involves  selective  consideration  of  alternative 
possible  states  of  affairs.  Metrics  of  performance  at  this  level  involve  logical, 
probabilistic,  and  explanatory  coherence  of  mental  models. 

2.  At  the  intermediate  level,  these  models  are  embedded  within  a  layer  of  critical 
questioning  which  motivates  the  generation  and  evaluation  of  possibilities.  Such 
dialogues  may  take  place  within  a  single  individual,  or  they  may  be  conducted  among 
different  individuals.  Critical  questioning  is  evaluated  by  reference  to  norms  for 
conducting  the  appropriate  kinds  of  critical  dialogue.  Dialogue  types  are 
differentiated  by  the  depth  of  probing  to  which  a  proponent  must  respond  and  the 
scope  of  the  permitted  responses. 

3.  At  the  outermost  layer,  critical  thinking  is  a  judgment  about  the  reliability  of  a 
cognitive  faculty,  hence,  the  degree  of  trust  that  should  be  placed  in  its  outputs.  The 
critical  dialogue  is  only  one  of  various  available  cognitive  or  social  processes  that 
might  be  utilized  to  generate  beliefs  and  decisions.  Different  processes,  such  as 
pattern  recognition,  may  be  more  reliable  under  some  conditions. 


In  sum,  critical  thinking  skill  is  exemplified  by  asking  and  answering  critical  questions  about 
alternative  possible  states  of  affairs,  to  the  extent  that  such  questioning  is  likely  to  increase  the 
reliability  of  the  overall  activity  in  achieving  its  purpose . 


FIGURE  1:  A  model  of  critical  thinking  with  three  embedded  layers:  mental  models,  critical 
dialogue,  and  control  based  on  reliability. 
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In  the  remainder  of  this  article,  we  will  very  briefly  discuss  some  of  the  background  and 
rationale  for  this  theory,  and  return  at  the  end  to  the  question  of  usefulness  in  Army  battlefield 
decisions. 

AVOIDING  THE  PITFALLS  OF  INTELLECTUALISM 

Modem  philosophy  began  (e.g.,  Descartes,  Locke,  Hume)  with  the  notion  that  we  have  a 
duty  to  carefully  decide  whether  to  accept  or  reject  our  beliefs,  and  a  duty  to  base  those  decisions 
upon  good  evidence.  At  the  beginning,  philosophers  thought  evidence  was  good  only  if  it 
rendered  a  conclusion  absolutely  certain.  Now,  philosophers  acknowledge  uncertainty,  and  good 
evidence  only  needs  to  provide  sufficient  justification  (whatever  that  is).  But  even  so,  a 
consistent  underlying  point  of  view  has  persisted  and  has  had  an  enormous  influence  on  the 
critical  thinking  movement.  Even  if  our  beliefs  are  true,  unless  we  accepted  them  on  the  basis  of 
what  we  explicitly  take  to  be  good  evidence,  we  are  correct  only  by  a  lucky  accident  (P.  Klein, 
2000). 

Example 

Suppose  MAJ  Jones  correctly  believes  that  there  is  an  enemy  T-62  tank  in  the  vicinity. 
Suppose  that  she  believes  it  because  she  saw  the  tank  and  is  in  fact  highly  accurate  in 
recognizing  types  of  tanks.  MAJ  Jones,  however,  does  not  believe  that  she  is  sufficiently  skilled 
to  identify  a  T-62.  Does  MAJ  Jones  know  that  the  tank  is  a  T-62  under  either  of  these 
circumstances?  The  intellectualist  viewpoint  would  say  no.  Even  though  she  accepts  the  belief 
based  on  good  evidence,  she  does  not  take  the  evidence  to  be  good.  Thus,  she  was  right  about  the 
tank  by  accident. 

From  this  point  of  view,  the  purpose  of  critical  thinking  is  to  ensure  that  we  have  explicit 
reflective  knowledge  of  all  our  first-level  beliefs,  our  reasons  for  accepting  them,  and  the  criteria 
that  determine  whether  the  reasons  are  sufficient.  Sosa  (1991,  p.  195)  dubbed  this  view  the 
“intellectualist  model  of  justification.”  But  is  this  the  best  view  of  what  critical  thinking  is  all 
about? 


Siegel  (1997)  falls  well  within  the  intellectualist  tradition  when  he  says  that 

...being  a  critical  thinker  requires  basing  one’s  beliefs  and  actions  on  reasons... 
the  beliefs  and  actions  of  the  critical  thinker,  at  least  ideally,  are  justified  by 
reasons  for  them  which  she  has  properly  evaluated  (p.14;  italics  in  original). 

This  view  appears  everywhere  in  the  critical  thinking  literature,  to  the  point  where  it  seems  to  be 
nothing  more  than  simple  common  sense.  Not  surprisingly,  the  notion  of  argument  (i.e.,  reasons 
for  conclusions)  is  central  in  textbooks  and  theoretical  discussions  of  critical  thinking. 
Unfortunately,  if  applied  universally  and  consistently  as  Siegel  (1997:  p.  16)  says  it  should  be, 
the  demand  for  argument  raises  the  danger  of  an  infinite  regress  (Dancy  &  Sosa,  1992: 
p.  209-212).  If  reasons  are  required  for  every  belief,  then  reasons  must  be  provided  to  justify  the 
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reasons,  to  justify  the  reasons  of  those  reasons,  and  so  on.  Critical  thinking  may  never  come  to  an 
end.  There  are  only  three  ways  to  respond  to  this  problem  within  the  philosophical  tradition:  The 
list  of  reasons  is  infinite,  it  circles  back  on  itself,  or  it  stops.  If  it  stops,  the  point  at  which  it  stops 
may  be  arbitrary  or  non-arbitrary. 


Figure  2:  Foundationalist  paradigm  for  acceptability  of  beliefs:  a  pyramid.  Arrows  represent  arguments  in 
which  conclusions  are  inferred  from  reasons.  Every  chain  of  argument  must  be  traceable  back  to  basic 
beliefs,  which  are  anchored  in  experience  or  logical  intuition  (shaded  boxes). 

If  the  list  of  reasons  continues  down  infinitely  without  ever  reaching  bottom,  conclusions 
can  never  be  justified.  The  result  is  skepticism  about  the  possibility  of  knowing  anything  (e.g., 
Ungar,  1974;  Foley,  1990).  A  variant  of  skepticism  is  that  the  list  of  reasons  ends  with  arbitrary 
assumptions  that  provide  the  basis  for  the  other  beliefs.  This  is  the  relativist  position,  that  beliefs 
are  not  justified  absolutely,  but  only  relative  to  the  framework  of  assumptions  that  happens  to  be 
accepted  in  a  domain  or  culture,  or  even  by  a  specific  individual  in  a  specific  context.  But  it  is 
hard  to  make  a  case  for  the  usefulness  of  critical  thinking  if  there  are  no  intersubjective  standards 
of  reasoning. 

A  second  possibility  is  that  the  chain  of  reasons  eventually  circles  back  on  itself.  For 
example,  continuing  down  the  chain  of  reasons,  we  would  eventually  arrive  again  at  the 
conclusion.  Siegel,  like  many  critical  thinking  theorists  and  informal  logicians,  rejects  the  idea 
that  a  chain  of  arguments  can  legitimately  circle  back  on  itself.  As  he  sees  it  (Siegel,  1 997: 
p.  71),  such  an  argument  commits  the  fallacy  of  begging  the  question ,  in  which  the  reasons  for  a 
conclusion  turn  out  to  contain  the  conclusion  itself.  In  other  words,  the  reason  for  accepting  p  is, 
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ultimately,/?  itself.  Siegel  (along  with  most  other  theorists  in  critical  thinking  and  informal  logic) 
is  therefore  committed  to  the  third  and  most  ambitious  possibility,  that  the  list  of  reasons  must 
come  to  rest  on  solid  ground,  with  beliefs  that  do  not  themselves  require  reasons  and  which  can 
serve  as  foundations  for  other  beliefs  (Figure  2).  These  beliefs  must  be  distinguished  by  some 
intrinsic  cognitively  accessible  feature  that  lends  them  a  higher  level  of  certainty,  such  as  their 
origins  in  perception  or  logic.  That  view  is  called  foundationalism  (Chisholm,  1977;  Pollock  & 
Cruz,  1999). 

COHERENCE 

Unfortunately,  foundationalists  have  been  unable  to  successfully  define  a  convincing 
class  of  basic  beliefs  for  which  arguments  are  unnecessary.  Virtually  every  belief  depends  in 
some  way  on  other  beliefs  for  its  justification.  For  that  reason  and  others,  many  philosophers 
urge  consideration  of  a  more  sophisticated  variant  of  the  “circular  reasoning”  option  called 
coherentism  (Thagard,  2000;  BonJour,  1985;  Lehrer,  2000;  Harman,  1986;  Quine  &  Ullian, 

1970;  Everitt  &  Fisher,  1995).  Coherentists  accept  that  a  chain  of  arguments  for  a  conclusion 
will,  if  pursued  long  enough,  arrive  back  at  the  conclusion  itself,  just  as  a  chain  of  dictionary 
definitions  will  eventually  arrive  back  at  the  original  word.  An  explanatory  hypothesis  draws 
support  from  the  observations  that  it  explains,  but  also,  the  veracity  of  the  observations  is 
supported  by  the  existence  of  a  good  explanation.  In  short,  the  premises  in  an  argument  are  not 
“basic”  in  any  deep  sense  that  differentiates  them  from  the  conclusion.  “Arguments”  might  run 
in  either  direction. 
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Example 

Suppose  MAJ  Smith  believes  that  she  saw  a  tank.  Since  a  tank  is  an  easily  recognized 
object  and  visibility  conditions  are  excellent,  this  is  a  good  candidate  for  a  basic  belief.  But  it  can 
be  undermined  if  it  turns  out  to  clash  with  other  beliefs  which  on  the  face  of  it  seem  less  secure. 
Suppose  MAJ  Smith  learns  that  the  enemy  has  deployed  dummy  tanks  in  the  region,  or 
remembers  that  the  area  where  she  “saw”  the  tank  is  shown  as  a  swamp  on  the  map.  These  non- 
basic  beliefs  may  trump  her  confidence  in  the  perceptual  judgment.  Alternatively,  the  perceptual 
judgment  might  lead  MAJ  Smith  to  question  the  map  or  the  reports  of  dummy  tanks.  MAJ  Smith 
must  determine  which  overall  set  of  beliefs  is  most  plausible,  including  beliefs  about  the 
presence  of  the  tank,  the  accuracy  of  the  map,  the  reliability  of  the  reports  about  dummy  tanks, 
and  the  reliability  of  his  own  perceptual  judgment.  In  other  words,  MAJ  Smith  must  evaluate  the 
plausibility  of  alternative  mental  models.  The  decision  whether  there  is  a  tank  will  depend  on 
general  beliefs  about  the  accuracy  of  maps,  intel  reports,  and  perceptual  experiences,  which  in 
turn  depend  in  part  on  the  past  performance  of  similar  maps,  reports,  and  perceptions.  That  is,  the 
selection  of  a  plausible  mental  model  will  depend  on  its  coherence  with  a  larger  body  of  beliefs, 
each  of  which  is  justified  with  respect  to  the  others  by  the  same  set  of  coherence  relationships. 

Reasoning  may  be  circular  if  carried  on  long  enough,  but  coherentists  deny  that 
justification  is  circular  because  they  reject  the  foundationalist  equation  of  justification  with 
reasoning.  Justification  is  not  directly  transferred  from  one  belief  to  another  by  a  linear  series  of 
arguments  (Day,  1989).  From  the  coherentist  perspective,  it  is  the  system  of  beliefs  that  is  the 
target  of  justification,  not  the  individual  beliefs  within  it  (Figure  3).  A  system  of  beliefs  is 
coherent  when  its  members  are  tightly  interconnected  by  logical,  conceptual,  explanatory,  or 
other  such  relationships.  Every  belief  contributes  some  support  to  every  other  belief  and  in  turn 
draws  support  from  every  other  belief,  just  as  each  stone  in  an  arch  depends  on  the  other  stones. 
Arguments  bear  on  justification  indirectly,  by  exposing  inferential  relationships  that  contribute  to 
the  coherence  of  the  system  of  beliefs  as  a  whole.  An  individual  belief  is  justified  indirectly  by 
having  a  place  in  such  a  coherent  system  of  beliefs.  Even  perceptual  beliefs,  which  were  not 
acquired  by  inference  from  other  beliefs,  are  justified  because  reasons  could  be  given,  e.g.,  by 
citing  the  reliability  of  visual  processes  under  good  conditions  of  visibility.  Arguments  are 
essential  tools,  since  they  may  be  used  to  show  that  a  target  belief  coheres  with  other  beliefs  that 
have  already  been  accepted.  But  clearly,  arguments  for  individual  beliefs  have  a  much 
diminished  role  in  settling  questions  of  justification. 

We  have  seen  that  a  central  problem  of  critical  thinking  is  how  to  know  when  to  stop 
demanding  reasons  for  a  belief.  Some  possible  answers  are: 
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■  Skepticism:  Never — -justification  cannot  be  completed. 

■  Relativism:  At  assumptions  that  cannot  themselves  be  justified. 

■  Foundationalism:  At  a  rock-bottom  set  of  beliefs,  based  on  sense  perception  or  logic, 
that  do  not  require  justification  in  terms  of  other  beliefs. 

■  Coherentism:  At  any  already  accepted  members  of  a  coherent  system  of  beliefs. 

None  of  these  positions  is  altogether  satisfactory.  On  the  one  hand,  as  we  mentioned, 
foundationalism  fails  because  all  beliefs  depend  on  other  beliefs.  A  second  problem  with 
foundationalism  is  that  it  fails  to  explain  how  to  choose  between  plausible  arguments  that  lead  to 
conflicting  conclusions.  (This  is  also  a  problem  with  informal  logic,  as  discussed  in  Cohen, 
Salas,  &  Riedel,  2001.)  On  the  other  hand,  coherentism  has  the  opposite  problems:  First,  some 
beliefs  do  in  fact  receive  priority  over  others,  even  if  they  are  not  known  with  certainty; 
examples  include  observation  reports  about  nearby  objects  in  plain  sight.  But  how  can  this 
priority  be  explained  if  every  belief  is  justified  in  the  same  way,  by  its  place  within  the  same 
system  of  beliefs?  Second,  coherentism  provides  an  account  of  how  conflicting  arguments  are 
resolved,  via  the  evaluation  of  alternative  systems  of  belief.  But  even  for  moderately  sized  belief 
systems,  the  combinatorics  of  inference  far  exceed  human  cognitive  capabilities  (Cherniak, 
1986). 


figure  3:  Coherentist  paradigm  for  acceptability  of  beliefs:  a  network.  The  system  of  beliefs  is  justified 
as  a  whole  by  the  inferential  links  among  its  components  and  its  overall  simplicity  and  comprehensiveness. 
Beliefs  are  not  classified  into  types  with  different  epistemological  status,  such  as  basic  or  not  basic. 
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There  are  two  kinds  of  responses  to  problems  with  pure  coherentism,  one  of  which  makes 
intellectualism  worse,  while  the  other  makes  a  dramatic  break  with  it.  The  intellectualist 
solution  (BonJour,  1985;  Lehrer,  2000;  Harman,  1973)  requires  a  higher  degree  of  reflective 
self-awareness,  stipulating  that  the  system  of  beliefs  be  evaluated  in  terms  of  coherence  with 
second-tier  beliefs  about  the  origins  and  reliability  of  all  its  first-order  beliefs.  Thus,  the  priority 
of  perceptually  based  beliefs  is  a  consequence  of  the  coherence  of  meta-beliefs  about  how 
reliable  perception  is  under  relevant  conditions.  This  requirement  constitutes  an  admission  that 
pure  coherence  is  insufficient  to  support  the  justification  of  beliefs.  Specific  kinds  of  beliefs  (i.e., 
second-order  beliefs  about  reliability)  must  be  part  of  the  mix.  However,  the  demand  for 
continuous  reflective  awareness  exceeds  human  capabilities.  It  also  threatens  another  kind  of 
vicious  regress,  involving  beliefs  about  beliefs,  beliefs  about  those  beliefs,  and  so  on,  unless  it 
reverts  to  a  form  of  foundationalism  in  which  beliefs  about  reliability  are  the  unquestioned 
foundations  in  need  of  no  further  arguments  (Sosa,  1991,  pp.  205-207). 

RELIABILITY 

The  more  appealing  solution  is  to  accept  that  some  of  the  factors  justifying  belief 
acceptance  may  not  be  cognitively  accessible.  First,  perceptual  systems  may  reliably  anchor  a 
system  of  beliefs  in  reality  even  if  the  subject  has  no  explicit  reflective  awareness  of  their 
reliability.  Second,  coherence  may  be  established  by  relatively  automatic  processes  of  spreading 
activation  across  a  network  of  beliefs,  rather  than  as  a  result  of  deliberate  reasoning  (Thagard, 
2000;  Cohen,  Thompson,  Shastri,  Salas,  Freeman,  Adelman,  2000b).  The  mutual  influence  of 
beliefs  will  depend  on  the  distance  that  activation  must  travel  in  the  network.  The  role  of 
deliberate  critical  thought,  on  the  other  hand,  is  more  limited:  It  will  selectively  activate  and 
evaluate  modular  subsets  of  beliefs,  i.e.,  mental  models  (Kornblith,  1989;  Cohen,  Thompson, 
Shastri,  Salas,  Freeman,  Adelman,  2000a).  These  responses  imply  that  a  belief,  whether 
perceptual  or  inferential,  may  constitute  genuine  knowledge  even  though  the  cognizer  is  unable 
to  articulate  reasons  for  holding  it.  This  idea  is  called  externalism.  (The  idea  that  our  evidence 
for  beliefs  must  be  conscious  or  readily  made  conscious  is  called  internalism,  and  is  shared  by 
both  foundationalism  and  coherentism.) 
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FIGURE  4:  Reliabilist  paradigm  for  acceptability  of  beliefs:  a  series  of  input/output  processes.  Beliefs  are 
justified  to  the  extent  that  they  are  produced  or  sustained  by  processes  that  reliably  achieve  the  goal  of 
accepting  true  beliefs  and  avoiding  false  beliefs  under  relevant  environmental  conditions.  In  the  version 
shown  here,  inputs  to  perceptual  faculties  come  from  the  environment,  while  inputs  to  memory  and 
inferential  faculties  include  the  outputs  of  other  belief-generation  processes. 

Extemalism  has  attracted  considerable  recent  interest  from  philosophers  (e.g.,  Goldman, 
1992,  1986;  Dretske,  1983;  Nozick,  1981;  Sosa,  1991;  Plantinga,  1993).  According  to  one 
version  of  extemalism,  called  reliabilism  (Nozick,  1981),  a  belief  is  justified  if  it  is  generated  or 
sustained  by  a  method  that  “tracks”  the  truth,  i.e.,  tends  to  produce  the  belief  that  p  if  and  only  if 
p  is  true.  According  to  another  variant  of  reliabilism  (Goldman,  1992,  1986),  a  belief  is  justified 
if  it  is  generated  or  sustained  by  cognitive  processes  that  reliably  generate  truths  and  avoid 
falsehoods  under  the  relevant  conditions.  Beliefs  of  different  kinds  are  more  or  less  justified 
depending  on  the  processes  and  mechanisms  that  produced  them  and  the  specific  conditions 
under  which  the  processes  were  operating  (Figure  4). 

Extemalism  does  not  insist  that  a  person  have  cognitive  access  to  reasons  for  a  belief,  that 
a  person  have  second-order  beliefs  about  the  reliability  of  first-order  beliefs,  or  even  that  beliefs 
are  always  under  voluntary  control.  A  person  is  deemed  expert  or  non-expert  based  on 
performance  and  results:  the  actual  accuracy  of  her  judgments  under  various  conditions. 
Extemalism  accounts  for  our  willingness  to  attribute  knowledge  to  people  even  when  they 
cannot  accurately  articulate  the  reasons  for  their  judgments  (Sternberg  &  Horvath,  1999;  Berry  & 
Dienes,  1993;  Nisbett  &  Wilson,  1977).  There  is  evidence  that  experts  can  become  highly 
proficient  in  recognitional  skills  in  which  they  are  less  able  than  novices  to  describe  their  own 
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thought  processes.  For  example,  expert  physicians  are  sometimes  not  able  to  retrieve  the 
explanation  supporting  a  diagnosis  (Patel,  Arocha,  &  Kaufman,  1999,  p.  82).  Extemalism  allows 
evaluation  of  a  belief  in  terms  of  the  objective  effectiveness  of  strategies  in  the  external 
environment,  relatively  automatic  processes  (such  as  perception,  pattern  recognition,  and 
constraint  satisfaction  in  connectionist  networks),  and  features  of  cognitive  mechanisms  (such  as 
processing  capacity  and  the  structure  of  knowledge  in  long-term  memory).  It  thus  promises  more 
fundamental  integration  with  concerns  of  cognitive  psychology. 

Objections  to  reliabilism  have  stressed  several  points:  First,  there  is  the  coherence 
problem.  Judgments  about  reliability  must  be  part  of  a  network  of  beliefs  that  is  evaluated  with 
respect  to  its  coherence.  Thus,  there  is  no  escaping  the  kind  of  “circularity”  emphasized  by 
coherence  theories  (Sosa,  1991).  Coherence  theories  stress  the  coherence  of  reliability 

judgments,  while  externalist  theories  stress  the  reliability  of  judgments  based  on  coherence.  But 
which  is  primary? 

Second,  there  is  the  generality  problem.  The  reliability  of  a  cognitive  faculty  might  be 
thought  of  as  its  ratio  of  successes  to  failures  under  specified  circumstances.  But  then,  reliability 
depends  on  how  generally  or  specifically  the  circumstances  are  specified  (Conee  &  Feldman, 
2000).  If  they  are  specified  too  generally,  reliability  is  not  very  informative.  For  example, 
visually  formed  beliefs  seem  to  be  generally  reliable;  but  visual  pattern  recognition  processes 
that  identify  a  nearby  object  as  a  tank  in  good  conditions  are  much  more  reliable  than  the  average 
visually  formed  belief.  But  should  we  also  include  the  condition  that  dummy  tanks  exist  in  the 
area?  If  so,  that  same  process  is  less  reliable  than  the  average  visually  formed  belief.  If  we 
describe  the  actual  present  conditions  with  maximal  specificity,  then  reliability  reduces  to  truth 
or  falsity  of  the  belief  in  the  particular  case.  But  justification  should  not  entail  absolute  certainty; 
it  should  be  possible  to  have  a  justified  belief  that  is  false  or  an  unjustified  belief  that  is  true. 

How  then  is  the  appropriate  level  of  generality  chosen? 

The  third  and  final  problem  concerns  fairness  in  evaluation.  Recall  MAJ  Jones,  who  has  a 
highly  reliable  faculty  for  quickly  recognizing  different  types  of  tanks  as  a  result  of  long  training 
and  experience.  But  MAJ  Jones  does  not  realize  how  reliable  her  judgment  is  and  indeed 
believes  it  to  be  unreliable.  MAJ  Jones  would  seem  to  be  unjustified  in  accepting  her  own  beliefs 
about  tanks,  even  thought  they  are  reliable  (Bonjour,  1985).  Given  her  beliefs  about  her  own 
unreliability,  she  would  be  right  to  double  check  the  tank  identifications  before  accepting  them. 
Conversely,  recall  MAJ  Smith.  Her  faculty  for  recognizing  the  presence  of  a  tank  is  generally 
reliable,  but  is  unreliable  under  special  circumstances  (such  as  when  there  are  dummy  tanks  in 
the  area).  But  if  MAJ  Smith  had  no  way  of  knowing  that  dummy  tanks  were  in  the  area,  or 
indeed  had  reason  to  believe  there  were  none,  wouldn’t  her  tank  identifications  be  justified  even 
though  they  were  unreliable?  Both  of  these  points  have  been  taken  to  suggest  that  internalist 
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intuitions  based  on  fairness,  both  in  holding  people  responsible  for  errors  and  giving  them  credit 
for  successes,  are  not  accounted  for  by  external  ism. 

Solution  of  these  problems,  and  a  reconciliation  of  reliabilism  and  coherentism,  requires 
the  recognition  of  two  distinct  points  of  view:  the  person  whose  knowledge  is  being  assessed 
(call  her  the  proponent  P)  and  the  person  who  is  assessing  that  knowledge  (call  her  the  judge  J). 
Judgments  of  reliability  of  P  s  beliefs  are  made  by  the  assessor  J.  The  assessor’s  purpose  is  quite 
straightforward.  J  would  like  to  be  able  to  use  P’s  opinions  as  a  source  of  information  in  a 
particular  range  of  circumstances,  but  in  order  to  do  so  must  assess  the  extent  to  which  P’s 
beliefs  can  be  trusted  in  those  circumstances.  J  asks,  for  example:  Can  I  infer  from  the  fact  that 
MAJ  Jones  believes  this  tank  is  a  T-62  to  the  conclusion  that  it  is  a  T-62?  Can  I  infer  from  the 
fact  that  MAJ  Smith  believes  there  is  a  tank  in  the  vicinity  to  the  conclusion  that  there  is  a  tank  in 
the  vicinity?  J  would  like  to  infer  from  P’s  having  a  certain  belief,  that  the  belief  is  true  and  can 
be  justifiably  endorsed  and  adopted  by  J  herself  (Brandom,  2000,  p.  120). 

Distinguishing  these  two  points  of  view  enables  us  to  resolve  the  coherence  problem. 
From  the  point  of  view  of  the  assessor  J,  judgments  of  the  reliability  of  P  must  be  arrived  at  just 
as  other  judgments  are,  by  reference  to  their  coherence  with  J’s  other  beliefs  and  their  fit  to  J’s 
perceptual  experiences.  As  Brandom  puts  it,  concern  with  reliability  is  external  only  “because 
assessments  of  reliability  (and  hence  of  knowledge)  can  turn  on  considerations  external  to  the 
reasons  possessed  by  the  candidate  knower  [P]  himself.”  But  assessments  of  reliability  are  not 
external  to  the  reasons  possessed  by  the  assessor  J.  They  inevitably  occur  within  J’s  own  system 
of  beliefs,  and  coherence  with  those  beliefs  is  a  major  determinant  of  J’s  conclusions  regarding 
the  reliability  of  P.  Dual-perspective  reliabilism  takes  seriously  the  coherentist  conclusion: 
Second-order  beliefs  about  reliability  are  required  in  order  to  anchor  a  coherent  system  of  beliefs 
in  reality.  But  it  rejects  the  requirement  that  those  second-order  beliefs  be  part  of  the  same 
system  that  is  being  evaluated. 

Similarly,  the  generality  problem  arises  only  when  reliability  assessments  are  thought  of 
as  lacking  a  point  of  view,  hence,  as  independent  of  both  reasons  and  purposes.  Since  reliability 
is  assessed  from  J’s  perspective,  the  scope  of  reliability  assessments  will  depend  on  J’s  beliefs 
and  purposes.  In  particular,  reliability  assessments  will  depend  on  (a)  what  J  knows  about  the 
situation,  (b)  what  J  knows  about  P,  and  (c)  the  range  of  situations  in  which  J  might  want  to  trust 
P  as  a  source  of  information.  If  J  is  concerned  with  the  trustworthiness  of  MAJ  Smith’s 
perceptual  recognition  of  a  tank  and  is  aware  of  the  presence  of  dummy  tanks  in  the  area,  J  will 
not  regard  MAJ  Smith’s  judgment  as  reliable  evidence  for  the  presence  of  a  tank.  But  if  J  trusts 
MAJ  Smith  generally,  if  the  situations  where  dummy  tanks  are  present  constitute  a  small 
minority,  and  if  J  is  not  aware  of  the  presence  of  dummy  tanks  in  the  area,  then  J  will  justifiably 
conclude  that  MAJ  Smith’s  tank  report  is  reliable. 
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The  fairness  problem  is  in  part  a  matter  of  divergent  purposes  between  internalist  and 
externalist  points  of  view.  According  to  intemalism,  the  purpose  of  critical  thinking  is  to  fulfill 
an  intellectual  duty,  to  carry  out  one’s  intellectual  responsibilities  in  a  blameless  way.  Thus,  it  is 
unfair  to  blame  a  critical  thinker  for  disregarding  relevant  evidence  if  that  information  was  not 
cognitively  accessible  (It  is  also  unfair  to  credit  her  for  ignoring  evidence  that  was  cognitive 
accessible,  just  because  that  information  turned  out  to  be  inaccurate).  But  externalism  shifts  the 
purpose  of  critical  thinking:  It  emphasizes  the  bottom  line:  accepting  significant  true  beliefs  and 
rejecting  significant  false  ones.  Because  of  this  shift,  there  is  no  longer  an  issue  of  “fairness”  in 
allocating  praise  and  blame.  Nonetheless,  internalist  intuitions  about  fairness  can  be  captured  in 
an  externalist  account  by  considering  point  of  view.  The  candidate  knower  may  assess  the 
reliability  of  her  own  beliefs,  adopting  the  perspectives  both  of  assessor  J  and  of  subject  of 
assessment  P.  Intuitions  about  fairness  tend  to  correspond  to  the  point  of  view  of  the  candidate 
knower  when  evaluating  the  reliability  of  her  own  judgments.  From  MAJ  Smith’s  own  point  of 
view,  her  recognition  of  a  tank  is  reliable  because  she  believes  that  dummy  tanks  are  not  likely. 
And  since  MAJ  Jones  thought  her  tank  identifications  were  unreliable,  she  could  not  be  blamed 
for  seeking  further  verification  before  accepting  her  perceptual  judgments.  Both  MAJ  Smith  and 
MAJ  Jones  made  reasonable  decisions  based  on  the  reliability  assessments  they  made  about  their 

own  judgments.  J  reached  different  conclusions  simply  because  J  had  more  information  than  they 
did.  3 

But  if  the  two  perspectives  can  be  combined  within  the  same  person,  how  can  they 
remain  distinct?  Wouldn’t  reliability  judgments  be  identical  to  the  judgments  arrived  at  by  the 
first-order  process?  In  other  words,  if  a  reasoning  process  inferred  a  probability  of  .8  confidence 
in  a  conclusion,  wouldn’t  the  assessment  of  the  reliability  of  that  belief  also  have  to  be  .8,  if  it  is 
done  by  the  same  person?  The  answer  is  no.  The  reality  of  the  different  viewpoints  is  confirmed 
in  an  experimental  study  by  Leddo  and  Govedich  (1986),  in  which  different  points  of  view  were 
induced  by  assigning  different  roles  to  participants.  Participants  were  asked  to  estimate  the 
chance  of  success  of  a  battle  plan.  Participants  could  be  assigned  the  role  of  planners  or  of 
implementers.  When  participants  performed  as  implementers,  they  adopted  an  internalist  point  of 
view.  They  tended  to  estimate  the  chance  of  success  by  considering  the  possible  reasons  the  plan 
might  fail.  This  exercise  helped  them  anticipate  and  prepare  for  potential  problems  during  the 
execution  of  the  plan.  But  since  the  implementers  inevitably  overlooked  some  possibilities,  they 
overestimated  overall  chance  of  success.  When  participants  performed  as  plan  developers,  on  the 
other  hand,  they  adopted  an  externalist  point  of  view.  They  tended  to  estimate  chance  of  success 
statistically,  by  reference  to  the  past  frequency  of  success  in  plans  of  a  similar  kind,  not  by 
enumerating  failure  scenarios.  As  a  result,  planners  were  less  overconfident. 

The  two  points  of  view  are  distinct  even  when  they  are  both  embodied  in  the  same 
individual.  Critical  thinking  occurs  internally  by  challenging  a  thesis  or  plan  and  making 
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adjustments  in  response  to  problems  that  are  found.  In  the  internalist  sense,  critical  thinking  is  an 
intrinsic  part  of  reasoning.  But  critical  thinking  occurs  externally  by  stepping  back  and 
questioning  the  reliability  of  the  process  as  a  whole  under  relevant  conditions,  in  order  to  select 
the  appropriate  process,  regulate  its  use  of  resources,  and  determine  when  confidence  in  the 
conclusion  is  high  enough  to  stop.  Since  this  kind  of  evaluation  is  done  “from  the  outside,”  the 
process  being  evaluated  may,  but  need  not  itself  involve  reasoning;  instead  it  might  concern  the 
accuracy  of  a  perception,  recall,  or  recognition.  The  two  viewpoints  draw  on  different  kinds  of 
information  and  involve  different  attitudes.  They  correspond  to  distinct  but  equally  important 
levels  of  critical  thinking. 

RELIABILITY  AND  COHERENCE  IN  CRITICAL  THINKING 

Critical  thinking  research  and  teaching  has  paid  scant  attention  to  non-foundationalist 
viewpoints  (Freeman,  2000).  This  is  the  reason  that  the  concept  of  argument  (with  individual 
beliefs  as  conclusions)  has  occupied  center  stage.  Non-foundationalist  approaches  such  as 
coherentism  and  reliabilism,  shift  the  emphasis  away  from  deliberative  arguments  about 
individual  beliefs.  Coherentism  accounts  well  for  the  mutual  adjustment  of  beliefs  to  one  another 
in  networks,  but  not  for  the  special  role  of  perceptual  inputs  or  for  computational  limitations. 
Reliabilism  accounts  for  beliefs  in  terms  of  the  specific  cognitive  faculties  that  generate  or 
sustain  them,  including  both  perceptual  and  inferential  systems  as  they  operate  in  real 
environments. 

The  three-part  model  of  critical  thinking  (Figure  1)  integrates  insights  from  coherentist 
and  reliabilist  theories  of  justification.  The  version  of  reliabilism  depicted  in  Figure  4  has  a 
foundationalist  flavor  because  reasoning  builds  on  a  distinct,  privileged  class  of  beliefs  generated 
by  perception.  By  contrast,  Figure  5  is  a  reliabilist  framework  that  incorporates  both  coherentism 
and  critical  thinking.  No  beliefs  are  immune  to  revision  based  on  incoherence  with  other  beliefs. 
Perceptual  systems  produce  experiences  rather  than  beliefs,  and  these  experiences  are  causal 
inputs  to  belief  generating  faculties.  In  other  words,  Figure  5  rejects  the  foundationalist 
assumption  that  there  is  a  privileged  class  of  beliefs  that  is  immune  to  reasoning.  On  the  other 
hand,  it  acknowledges  that  perceptual  experience  is  an  essential  input  to  a  coherence-based  belief 
system  (c.f.,  Haack,  1993;  Thagard,  2000).  The  role  of  beliefs  that  are  closely  related  to 
perceptual  experiences  is  explained  by  appeal  to  their  reliability,  but  it  is  not  necessary  for  the 
candidate  knower  herself  to  have  reflective  second-order  beliefs  about  her  first-order  beliefs,  as 
coherence  theories  require. 

The  three-part  model  of  critical  thinking  forms  the  top  tier  of  Figure  5,  consisting  of 
critical  dialogue  about  mental  models  to  achieve  purposes  under  specific  environmental 
conditions.  Although  critical  thinking  is  reflective,  it  interacts  with  the  more  automatic  operation 
of  the  coherence  system.  It  takes  sets  of  beliefs  from  the  coherence  system  as  inputs,  creates  and 
critically  evaluates  mental  models,  and  in  turn  feeds  its  conclusions  back  as  inputs  to  the 
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coherence  system.  All  cognitive  faculties— perception,  coherence-based  reasoning,  and  critical 
thinking  are  designed  to  reliably  achieve  particular  purposes  in  particular  environments  in 
consort  with  each  other.  Judgments  of  reliability  may  be  made  from  an  external  point  of  view,  to 
determine  whether  another  person’s  opinions  can  be  trusted,  or  may  be  made  internally  (but  still, 

from  a  hypothetical  outside’  point  of  view)  to  regulate  use  of  one’s  own  faculties  in  knowledge 
acquisition. 

We  will  now  discuss  in  a  bit  more  detail  how  the  components  of  this  model  work  together 
in  critical  thinking. 


FIGURE  5:  A  reliabilist  framework  that  integrates  a  coherence  theory  of  reasoning  with  the  critical  thinking 
model  in  Figure  1 . 

MENTAL  MODELS 

Johnson-Laird  (1983;  Johnson-Laird  &  Byrne,  1991)  cites  evidence  that  humans  reason 
not  (or  not  only)  in  terms  of  syntactic  formal  patterns  but  in  terms  of  meaning.  Comprehending 
an  assertion  includes  understanding  what  possible  states  of  affairs  are  compatible  with  the 
assertion  and  which  are  excluded  (Johnson-Laird  &  Byrne,  1991).  Inference  is  in  large  part  a 
process  for  comprehending  multiple  assertions,  that  is,  for  determining  what  states  of  affairs  are 
consistent  with  several  different  assertions.  The  conclusion  of  an  inference  must  be  true  in  every 
surviving  possibility. 

The  representation  of  a  possible  state  of  affairs  is  called  a  mental  model.  Typically,  a 
simple  statement  (e.g.,  the  enemy  will  attack  through  the  northern  pass )  is  compatible  with  many 
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different  states  of  affairs  (when  and  how  they  will  attack),  but  people  often  use  a  single, 
representative  mental  model  to  conserve  processing  capacity  (Johnson-Laird  &  Byrne,  1991,  p. 
170).  They  change  the  representation  or  expand  it  to  include  other  possibilities  only  when  forced 
to  do  so.  When  a  sentence  contains  logical  connectives,  such  as  and,  or,  all,  and  some,  people 
use  knowledge  of  the  meaning  of  such  connectives  to  construct  some  or  all  of  the  appropriate 
mental  models.  Suppose  MAJ  House  and  MAJ  Kerr  are  analyzing  enemy  intent,  and  MAJ  House 
says: 

MAJ  House:  The  enemy  will  attack  either  through  the  northern  pass  or  the  southern 

pass. 

Neither  MAJ  House  nor  MAJ  Kerr  believes  the  enemy  will  attack  through  both  passes.  The  or 
statement  thus  suggests  only  two  possible  mental  models:  one  in  which  the  enemy  attacks  in  the 
south  (and  not  the  north),  and  the  other  in  which  they  attack  in  the  north  (and  not  the  south): 


1 

The  enemy  will  attack 
through  the  south 

NOT:  The  enemy  will 
attack  through  the  north. 

2 

NOT:  The  enemy  will 
attack  through  the  south 

The  enemy  will  attack 
through  the  north. 

To  conserve  processing  capacity,  according  to  Johnson-Laird,  and  Byrne,  people  usually  do  not 
represent  negations  unless  they  are  stated  explicitly.  So,  the  two  cells  with  NOT  would  be  left 
blank. 


Now  suppose  MAJ  Kerr  says,  “I don ’t  believe  it  will  be  the  northern  pass."  MAJ  Kerr 
has  committed  herself  to  mental  model  #1 : 


1 

The  enemy  will  attack 

through  the  south. 

MAJ  House,  however,  is  still  uncertain,  and  wants  to  know  the  reasons  for  this  conclusion.  MAJ 
Kerr  replies,  “ Because  they  don ’t  have  any  artillery  there."  Background  knowledge  of  enemy 
doctrine  suggests  that  the  enemy  will  not  attack  without  using  artillery  first  to  soften  up  the 
opposing  force.  Thus,  only  attack  in  the  south  is  consistent  with  the  absence  of  artilleiy  in  the 
north.  Hence,  MAJ  Kerr  has  concluded  that  the  enemy  will  attack  in  the  north.  The  following 
mental  models  capture  MAJ  Kerr’s  reasoning,  and  also  show  how  she  intends  to  persuade  MAJ 
House  to  accept  mental  model  #1 : 
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The  issue 

MAJ  Kerr’s  reason 

1 

The  enemy  will 
attack  through  the 
south. 

NOT:  The  enemy  has 
artillery  in  the  north. 

This  mental  model 
is  consistent  with 
background 
knowledge. 

2 

The  enemy  will 
attack  through  the 
north. 

NOT:  The  enemy  has 
artillery >  in  the  north. 

This  mental  model 
is  ruled  out  by 
background 
knowledge  of  likely 
enemy  tactics. 

In  this  example,  background  knowledge  ruled  out  mental  model  #2.  However,  Johnson-Laird  and 
Byrne  (1991)  show  how  such  conclusions  can  be  reached  by  more  explicit  deductive  reasoning 

with  mental  models.  Such  reasoning  is  typically  more  effortful.  For  example,  suppose  MAJ  Kerr 
had  said: 

MAJ  Kerr.  If  they  were  going  to  attack  in  the  north,  they  would  have  artillety 
nearby.  And  they  don ’t. 

Considering  only  the  simple  components  of  MAJ  Kerr’s  statements,  there  are  four  logical 
possibilities,  corresponding  to  different  combinations  of  truth  and  falsity  of  attack  in  the  north 
and  artillery  in  the  north.  The  conditional  statement  (If  they  were  going  to  attack  in  the  north, 
they  would  have  artillery  nearby)  excludes  just  one  of  these  four  situations: 


1 

NOT:  The  enemy  will 
attack  through  the  north. 

NOT:  The  enemy  has 
artillery  in  the  north. 

2 

The  enemy  will  attack 
through  the  north. 

NOT:  The  enemy  has 
artillery  in  the  north. 

This  situation  is 
excluded  by  the 

If _ then _ 

statement. 

3 

NOT:  The  enemy  will 
attack  through  the  north. 

The  enemy  has  artillery  in 
the  north. 

4 

The  enemy  will  attack 
through  the  north. 

The  enemy  has  artillery  in 
the  north. 

(We  omit  the  column  for  attack  in  the  south  for  simplicity.)  The  other  part  of  MAJ  Kerr’s 
argument  is  that  the  enemy  does  not  have  artillery  in  the  north.  Adding  that  piece  of  information 
excludes  mental  model  #3  and  mental  model  #4.  Thus,  the  only  surviving  possibility  is  mental 
model  #1,  and  the  conclusion  is  that  the  enemy  will  not  attack  in  the  north. 
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According  to  mental  model  theory,  the  difficulty  of  an  inference  increases  with  the 
number  of  alternative  possibilities  that  must  be  considered.  Thus,  the  use  of  background 
knowledge  (i.e.,  the  automatic  operation  of  a  coherence-based  network  of  beliefs)  to  eliminate  at 
least  some  of  the  possibilities,  as  in  our  example,  is  much  less  effortful  than  explicit  inference. 
Indeed,  a  major  advantage  of  mental  model  theory  over  other  approaches  is  that  it  can 
accommodate  both  automatic  and  deliberate  processes  in  any  mix.  Errors  in  explicit  inference 
may  occur  for  several  reasons:  the  number  of  possibilities  exceeds  capacity  limitations  of 
working  memory  (Johnson-Laird  &  Byrne,  1991,  p.  39);  there  is  a  tendency  to  represent  only 
explicit  and  true  components  of  premises  and  thus  to  neglect  possibilities  consisting  of  false 
components;  or  a  prior  tendency  to  believe  that  the  conclusion  is  correct  causes  the  reasoner  to 
cut  short  the  exploration  of  alternatives.  Of  course,  another  possible  cause  of  error  is  elimination 
of  possibilities  due  to  inaccurate  background  beliefs.  Because  of  such  limitations  and  biases, 
people  are  liable  sometimes  to  accept  a  conclusion  even  though  there  is  a  possible  state  of  affairs 
in  which  it  is  false. 

DIALOGUES 

The  field  of  informal  logic  has  lacked  a  unifying  theory  that  successfully  accounts  for 
different  types  of  arguments  and  the  errors  to  which  they  are  subject  (Walton,  1998,  p.  7).  A 
promising  approach,  which  is  drawing  increasing  attention,  is  the  interpretation  of  argument  as  a 
component  of  dialogue.  As  Johnson  (1996)  says,  “an  argument  understood  as  product — a  set  of 
propositions  with  certain  characteristics — cannot  be  properly  understood  except  against  the 
background  of  the  process  which  produced  it— the  process  of  argumentation.”  Dialogue  theorists 
attempt  to  describe  argumentation  by  means  of  rigorous,  idealized  models  of  interactive 
exchanges.  Such  models  specify  the  purposes  of  different  types  of  dialogue,  the  roles  that  are 
played  within  the  dialogue,  rules  for  each  player,  and  rules  for  determining  who  wins.  Actual 
discussions  can  be  analyzed  and  evaluated  in  terms  of  how  closely  they  approximate  the 
appropriate  paradigm  (Walton  &  Krabbe,  1995,  pp.  174-177). 

The  pragma-dialectical  theory  proposed  by  van  Eemeren  and  Grootendorst  (1994)  closely 
interweaves  normative  and  descriptive  elements.  An  ideal  of  critical  rationality  in  dialogue  is 
developed,  while  at  the  same  time  actual  processes  of  argumentative  discourse  are  studied 
empirically.  Actual  argumentative  discourse  is  reconstructed  from  the  perspective  of  the  ideal  of 
critical  discussion.  This  permits  the  discovery  of  practical  problems  or  errors  experienced  in 
argumentative  discourse,  and  forms  the  basis  for  development  of  appropriate  methods  in 
education  (van  Eemeren,  Grootendorst,  &  Snoeck  Henkemans,  1996).  The  source  of  the  norms  is 
an  ideal  of  actual  human  discourse,  rather  than  a  formal  axiomatic  system  (as  in  logic  or 
probability  theory).  According  to  Walton  (1996b),  “A  dialogue  is  a  goal-directed,  collaborative 
conversational  exchange,  of  various  types,  between  two  parties. ...  fallacy  is  defined  as  an 
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argument  or  a  move  in  argument  that  interferes  with  the  goal  of  a  dialogue  of  which  it  is 
supposed  to  be  a  part....” 

Dialogue  theory  provides  a  deeper  analysis  of  fallacies  than  the  usual  description  in  terms 
of  surface  features.  For  example,  one  fallacy  is  typically  described  as  attacking  the  person,  or  ad 
hominem.  A  simple  example  of  a  rule  of  discourse  emerging  from  dialogue  theory  is  the 
following:  ‘Parties  must  not  prevent  each  other  from  advancing  standpoints  or  from  casting 
doubt  on  standpoints.”  A/  hominem  fallacies  sometimes  involve  violation  of  this  rule.  According 
to  dialogue  theory,  when  personal  attacks  are  intended  to  prevent  an  opponent's  views  from 
being  fairly  considered,  the  violation  of  the  rule  of  cooperation  is  what  makes  this  an  error,  not 
surface  features  (“attacking  the  person”).  Other  fallacies  (e.g.,  argument  by  appeal  to  pity,  or 
threats  of  force)  that  described  very  differently  from  ad  hominem  appear  to  involve  violation  of 
the  same  dialogue  principle  and  thus  are  the  same  error  when  considered  at  a  deeper  level. 
Conversely,  in  other  contexts,  impugning  the  character  of  a  person  may  be  highly  appropriate, 
e.g.,  if  the  person’s  testimony  must  be  relied  on  in  drawing  a  conclusion.  Understanding  errors  in 
terms  of  dialogue  rules  provides  both  a  more  nuanced  and  a  more  accurate  assessment  of  their 
normative  status. 

Walton  (1998)  has  studied  a  variety  of  different  kinds  of  dialogue,  which  differ  in  their 
purposes  and  the  norms  by  which  they  are  conducted:  e.g.,  deliberation,  inquiry,  negotiation, 
information  seeking,  and  persuasion.  According  to  dialogue  theories,  participants  cooperate  to 
choose  the  type  of  dialogue  that  is  best  for  the  purpose  and  context  (van  Eemeren  & 

Grootendorst,  1992).  Hence,  they  must  make  reflective  judgments  about  the  relative  reliability  of 
different  dialogues  as  methods  for  achieving  their  goals.  They  must  also  reflectively  monitor 
adherence  to  the  norms  that  govern  the  relevant  type  of  dialogue  (Jackson,  1 989;  Johnson,  2000). 

According  to  van  Eemeren  and  Grootendorst  (1992,  pp.  34-37),  a  critical  discussion  is  a 
dialogue  type  used  for  the  resolution  of  a  difference  of  opinion.  Resolution  is  not  a  matter  of 
negotiation  (which  is  a  different  type  of  dialogue)  or  of  simply  setting  the  difference  aside.  It 
involves  persuading  one  of  the  parties  to  retract  doubt  concerning  the  other  party’s  position 
because  she  has  been  convinced  by  the  other  party’s  reasons,  or  conversely  for  one  of  the  parties 
to  relinquish  her  own  position  because  it  has  not  withstood  the  other  party’s  challenges.  In  some 
dialogue  models  (e.g.,  Walton,  1998;  van  Eemeren  and  Grootendorst,  1992),  there  are  two 
participants  or  roles:  a  proponent  and  an  opponent.  In  other  models  (e.g.,  Rescher,  1977),  there 
are  three  participants  or  roles:  a  proponent,  an  opponent,  and  a  judge.  Rules  governing  the 
possible  actions  of  each  participant  are  a  function  of  the  type  of  dialogue,  the  stage  of  the 
dialogue,  and  the  previous  statements  of  each  participant. 

In  the  confrontation  stage  of  a  critical  discussion,  a  difference  of  opinion  is 
acknowledged.  For  example,  the  proponent  expresses  a  standpoint  with  or  without  reasons,  and 
the  opponent  indicates  disagreement  or  expresses  doubt.  The  parties  may  also  seek  to  clarify  or 
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flesh  out  each  other’s  positions.  In  the  opening  stage,  which  is  likely  to  be  implicit  rather  than 
explicit  (van  Eemeren  and  Grootendorst,  1992,  p.  41),  the  parties  “agree”  on  the  type  of 
discussion  they  will  have  and  the  discussion  rules.  Specifically,  in  the  type  of  dialogue  called  a 
critical  discussion,  they  agree  that  one  will  take  the  role  of  proponent  and  the  other  will  take  the 
role  of  the  opponent.  The  proponent  incurs  an  obligation  to  defend  or  modify  her  standpoint  at 
each  move,  and  the  opponent  incurs  an  obligation  to  accept  or  reject  the  proponent’s  assertions  at 
each  move.  They  agree  that  each  assertion  must  support  the  goal  of  the  dialogue  type  they  have 
selected,  e.g.,  to  resolve  the  difference  of  opinion,  and  they  agree  not  to  shift  dialogue  types 
without  mutual  agreement.  They  also  agree  to  distinguish  which  assertions  are  meant  as 
conclusions  and  which  are  meant  to  be  reasons  for  those  conclusions. 

The  crucial  stage  of  a  critical  discussion  is  argumentation,  in  which  the  proponent  and 
opponent  carry  out  their  roles  of  defending  and  challenging  a  thesis,  respectively.  Normative 
models  of  this  stage  spell  out  the  types  of  assertions  that  are  permitted  to  each  side  as  a  function 
of  previous  assertions.  The  major  difference  between  the  proponent  and  opponent  in  a  critical 
dialogue  is  the  global  burden  of  proof.  It  is  up  to  the  proponent  to  create  a  positive  case  for  her 
standpoint.  The  opponent  merely  has  to  create  doubt.  (In  more  complex  types  of  dialogue,  the 
two  parties  may  defend  contrary  theses,  and  each  participant  in  effect  plays  opponent  to  the 
other.)  Although  the  global  burden  of  proof  is  static  (and  rests  upon  the  proponent),  as  each  side 
provides  arguments  or  challenges,  the  local  burden  of  proof  switches  back  and  forth  (Rescher, 
1977,  p.  27).  That  is,  whenever  either  side  advances  an  argument,  it  stands  until  explicitly 
rebutted  by  the  other  side.  In  the  concluding  stage  of  a  critical  discussion,  the  dispute  may  be 
ended  because  the  proponent  withdraws  her  thesis  or  because  the  opponent  withdraws  her  doubt. 

The  critical  discussion  (or  the  more  general  persuasion  dialogue  described  by  Walton, 
1998)  provides  a  promising  framework  for  both  understanding  and  training  critical  thinking.  The 
primary  reason  for  its  usefulness  is  the  functional  similarity  between  rationally  persuading 
another  individual  to  accept  or  reject  a  position,  and  rationally  determining  for  oneself  whether  a 
position  is  acceptable  or  not.  The  idea  of  a  dialogue  externalizes  necessary  functions  that  must 
take  place  within  an  individual  cognizer.  Thinking  may  be  fruitfully  studied  as  a  form  of  internal 
dialogue  in  which  a  single  individual  takes  on  distinct  dialectical  roles  (Walton,  &  Krabbe,  1995, 
p.  26).  Another  reason  for  focusing  on  dialogue  as  a  model  of  thinking  is  that  the  functional 
resemblance  between  thought  and  dialogue  is  more  than  a  coincidence.  A  variety  of 
developmental  psychologists  (starting  perhaps  with  Vygotsky)  have  proposed  that  thought  first 
develops  in  each  individual  as  internalized  speech  and  that  we  learn  to  reflect  on  and  evaluate 
our  own  thoughts  by  responding  to  the  thoughts  of  others  (Bogden,  2000).  As  noted  by  Rieke 
and  Sillars  (1997), 

...research  suggests  that  critical  thinking  is  really  a  mini-debate  that  you  carry  on 

with  yourself.  What  is  often  mistaken  for  private  thought  is  more  likely  an 
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“internalized  conversation”  (Mead),  an  “internal  dialogue”  (Mukarovsky),  or  an 

“imagined  interaction”  (Gotcher  and  Honeycutt). 

A  final  reason  for  interest  in  dialogue  theory  is  more  direct.  Much  critical  thinking  takes  place  in 
a  team  or  group  context,  in  which  dialogue  plays  a  literal  role  in  decision  making.  The  road  to 
improved  critical  thinking  in  both  an  individual  and  a  team  context  may  lead  through  training  in 
improved  skills  and  habits  for  critical  dialogue. 

MENTAL  MODELS  AND  DIALOGUES 

The  argumentation  stage  of  a  critical  dialogue  can  be  seen  as  a  process  of  constructing 
and  evaluating  mental  models.  Dialogue  theory  links  up  with  mental  model  theory  via  its  concept 
of  a  commitment  store  (Hamblin,  1970;  Rescher,  1977;  Walton  &  Krabbe,  1995).  According  to 
Hamblin  (p.  257),  “a  speaker  who  is  obliged  to  maintain  consistency  needs  to  keep  a  store  of 
statements  representing  his  previous  commitments,  and  require  of  each  new  statement  he  makes 
that  it  may  be  added  without  inconsistency  to  this  store...”  Walton  and  Krabbe  (1995) 
distinguish  two  kinds  of  explicit  commitment  stores:  commitments  based  on  assertions,  which 
the  speaker  is  obligated  to  defend,  and  commitments  by  a  listener  based  merely  on  concessions, 
which  the  listener  is  not  obligated  to  defend. 

Rules  for  permissible  moves  in  the  argumentation  stage  of  a  dialogue  refer  to  the  current 
status  of  these  commitment  stores,  and  specify  how  each  move  changes  their  contents.  The 
listener  can  challenge  any  assertion  by  the  speaker  as  long  as  that  assertion  is  not  in  the  listener’s 
own  assertion-based  commitment  store.  If  the  listener  challenges  a  commitment  based  on  an 
assertion  by  a  speaker  and  the  speaker  cannot  defend  it  by  supplying  reasons,  the  speaker  must 
retract  it.  When  the  listener  does  not  immediately  challenge  an  assertion  by  the  speaker,  the 
listener  has  conceded  it,  and  it  goes  into  the  listener’s  concession-based  commitment  store.  The 
listener  is  of  course  not  obligated  to  defend  her  concession,  but  must  allow  the  speaker  to  use  it 
in  argumentation  at  least  for  the  time  being.  The  listener  can  retract  the  concession  at  any  time 
simply  by  challenging  it,  as  long  as  it  is  still  in  the  speaker’s  commitment  store  (otherwise,  the 
challenge  would  be  irrelevant).  The  speaker  can  also  choose  to  retract  an  assertion  of  her  own, 
but  this  is  more  difficult  to  do  because  she  must  also  find  and  retract  any  other  commitments  that 
imply  the  retracted  assertion  (i.e.,  the  reasons  she  may  have  given  for  her  assertion).  If  there  are 
inconsistent  assertions  in  the  speaker’s  commitment  store,  and  the  listener  challenges  them,  then 
the  speaker  must  retract  at  least  one  of  the  conflicting  commitments  along  with  the  reasons  that 
led  to  it. 

Commitment  stores  are  simply  sets  of  mental  models.  Each  mental  model  in  the 
commitment  store  of  a  dialogue  participant  represents  a  state  of  affairs  that  is  regarded  as 
possible  by  that  participant  at  that  particular  time.  For  example,  in  the  argument  about  location  of 
attack,  MAJ  Kerr  is  the  proponent.  She  stated  a  thesis  (that  the  enemy  attack  will  be  in  the  south) 
which  MAJ  House  did  not  concede.  MAJ  House  thus  takes  the  role  of  opponent.  In  response  to 
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MAJ  House’s  challenge,  MAJ  Kerr  gave  a  reason  (lack  of  artillery  in  the  north)  that  was 
intended  to  persuade  MAJ  House  to  exclude  the  competing  possibility. 

Now  suppose  MAJ  House  concedes  that  absence  of  artillery  in  the  north  would  be  a 
good  reason  to  accept  MAJ  Kerr’s  conclusion,  if  the  reason  were  true.  (In  other  words,  she 
chooses  not  to  challenge  that  aspect  of  the  argument  for  the  time  being.)  But  MAJ  House 
expresses  doubt  about  the  reason  itself:  “  What  makes  you  think  there  is  no  artillery  in  the 
north?"  Since  MAJ  House  does  not  concede  the  truth  of  the  reason,  the  opponent’s  mental 
models  now  include  the  following  different  situations: 


Opponent 


The  issue 

Proponent’s  reason 

1 

The  enemy  will 
attack  through  the 
south 

NOT:  The  enemy  has 
artillery  in  the  north. 

2 

The  enemy  will 
attack  through  the 
north. 

NOT:  The  enemy  has 
artillery  in  the  north. 

Opponent  concedes 
that  this  possibility 
is  (temporarily) 
excluded 

3 

The  enemy  will 
attack  through  the 
south 

The  enemy  has 
artillery  in  the  north. 

Opponent  questions 
reason,  hence,  still 
believes  these  are 
possible. 

4 

The  enemy  will 
attack  through  the 
north. 

The  enemy  has 
artillery  in  the  north. 

Notice  that  the  opponent,  MAJ  House,  is  not  committed  to  the  negation  of  the  reason  (that  there 
is  artillery  in  the  north),  but  only  to  the  possibility  that  there  is  artillery  in  the  north.  Nor  is  the 
opponent  saying  that  the  conclusion  is  false.  But  she  is  saying  that  the  possibility  of  artillery  in 
the  north  opens  up  the  possibility  of  attack  in  the  north.  Specifically,  in  mental  model  #4  the 
enemy  has  artillery  in  the  north  and  attacks  through  the  north.  Since  the  proponent’s  conclusion 
is  not  true  in  all  the  possible  situations,  the  opponent  is  not  yet  convinced.  In  order  to  persuade 
the  opponent,  the  proponent  must  find  some  way  to  eliminate  mental  model  #4.  Suppose  that  the 
proponent  now  presents  a  reason  to  believe  that  the  enemy  has  no  artillery  in  the  north: 
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Proponent 


Issue 

Proponent’s  reason 

Proponent’s  reason 
for  the  reason 

1 

The  enemy  will 
attack  through  the 
south . 

The  enemy  has  no 
artillery  in  the  north. 

Our  imagery  of  the 
northern  sector  is 
excellent  and  shows 
no  artillery  in  the 
north. 

This  time  the  opponent,  MAJ  House,  concedes  the  truth  of  MAJ  Kerr’s  reason  (the 
imagery  is  good  and  showed  no  artillery  in  the  north).  But  now  she  wishes  to  go  back  and 
challenge  something  that  she  had  temporarily  conceded:  MAJ  Kerr’s  claim  that  absence  of 
artillery  is  in  fact  a  good  reason  under  these  circumstances  for  expecting  no  attack  in  the  north. 

In  particular,  MAJ  House  points  out,  “ Don ’t  we  have  reports  that  the  enemy  has  developed 
longer-range  artillery ?”  This  move  is  the  opposite  of  the  previous  one.  MAJ  House  accepts  the 
truth  of  the  reason  (no  artillery  in  the  north)  but  has  introduced  a  consideration  intended  to 
neutralize  or  cancel  it  out  as  evidence  for  the  conclusion  (no  attack  in  the  north).  The  absence  of 
artillery  in  the  north,  in  conjunction  with  the  fact  that  the  enemy  has  developed  longer  range 
artillery  is  not  evidence  for  the  conclusion  (since  the  enemy  could  use  artillery  located  at  a  longer 
distance).  We  call  such  an  objection  a  defeater.  The  opponent’s  mental  models  are  now  the 
following: 
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Opponent 


Issue 

Proponent’s 

reason 

Proponent’s 
reason  for  the 

reason 

Opponent’s 

defeater 

1 

The  enemy  will 
attack  through 
the  south. 

NOT:  The 
enemy  has 
artillery  in  the 
north. 

Our  imagery  of 
the  northern 
sector  is 
excellent  and 
shows  no 
artillery  in  the 
north. 

The  enemy  has 
developed  long 
range  artillery. 

2 

The  enemy  will 
attack  through 
the  north. 

NOT:  The 
enemy  has 
artillery  in  the 
north. 

Our  imagery  of 
the  northern 
sector  is 
excellent  and 
shows  no 
artillery  in  the 
north. 

The  enemy  has 
developed  long 
range  artillery. 

... 

... 

The  three  dots  indicate  that  the  opponent  has  merely  conceded  the  imagery  evidence  and  thus  the 
absence  of  artillery  in  the  north,  but  is  not  committed  to  defending  them.  She  is  at  liberty  to 
challenge  them  again  later.  We  have  used  Johnson-Laird's  (1983)  convention  for  representing 
implicit  mental  models  as  a  handy  way  to  represent  concessions  in  a  commitment  store. 

The  opponent  s  challenge  to  MAJ  Kerr’s  argument  introduces  the  very  important  topic  of 
defeasibility,  and  shows  how  dialogue  theory  and  mental  models  in  conjunction  help  clarify 
some  key  aspects  of  reasoning  about  uncertainty.  Informal  logicians,  psychologists, 
philosophers,  and  artificial  intelligence  researchers  generally  agree  that  non-deductive  inferential 
conclusions  are  subject  to  defeat  by  new  information,  i.e.,  such  inferences  are  defeasible.  As  we 
have  seen,  a  defeater  (e.g.,  the  development  of  longer  range  artillery)  may  undermine  an 
inference  without  providing  evidence  for  the  opposite  conclusion.  Notice  in  addition  that  the 
opponent  does  not  even  deny  that  the  original  evidence  (the  absence  of  artillery  in  the  north) 
supported  the  proponent’s  conclusion  (no  attack  in  the  north).  The  opponent  merely  points  out 
that  while  no  artillery  generally  indicates  no  attack,  there  are  special  circumstances  in  this 
situation  that  must  be  taken  into  account.  The  further  bit  of  information  about  longer  range 
artillery  neutralizes  the  support  given  by  the  proponent’s  evidence  for  the  proponent’s 
conclusion  in  this  context.  At  this  stage  of  the  dialogue,  MAJ  Kerr’s  argument  is  defeated  and  no 
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conclusion  about  location  of  attack  can  be  drawn.  Thus,  she  may  have  to  retract  her  conclusion 
about  attack  in  the  south. 

Defeasibility,  however,  is  an  open-ended  aspect  of  reasoning  about  the  real  world.  Thus, 
MA.l  Kerr  may  answer  MAJ  House’s  challenge  by  defeating  the  defeater.  For  example,  MAJ 
Ker.  tplies.  That  may  well  be,  but  I  don  t  recall  any  indications  that  they ’ve  deployed  the  new 
systems  yet.”  If  the  enemy  has  not  deployed  the  new  artillery,  then  mere  development  of  the 
technology  is  irrelevant.  The  original  argument  based  on  lack  of  artillery  in  the  north  regains  its 
former  force.  The  proponent’s  commitment  store  still  has  only  one  explicit  mental  model: 

Proponent 


Issue 

Proponent’s 

reason 

Proponent’s 
reason  for 
the  reason 

Opponent’s 

defeater 

Proponent’s 
defeater  of 
the  defeater 

1 

The  enemy 
will  attack 
through  the 
south. 

NOT:  The 
enemy  has 
artillery  in 
the  north. 

Our  imagery 
of  the 
northern 
sector  is 
excellent  and 
shows  no 
artillery  in 
the  north. 

The  enemy 
has 

developed 
long  range 
artillery. 

The  enemy 
has  not 
deployed  the 
new  artillery. 

... 

The  three  dots  show  that  MAJ  Kerr  has  conceded  the  development  of  longer  range  artillery  but 
may  choose  to  make  alternative  possibilities  explicit  later.  In  this  mental  model,  it  is  clear  that 
the  three  claims  in  combination— lack  of  artillery  in  the  north,  and  possession  of  longer  range 
artillery  that  has  not  been  deployed— do  provide  evidence  against  attack  in  the  north.  Thus,  only 
one  possibility  survives,  mental  model  #1,  in  which  the  enemy  attacks  in  the  south.  MAJ  Kerr’s 
original  conclusion  has  been  vindicated — unless  of  course  the  opponent  comes  up  with  another 
challenge,  to  which  MAJ  Kerr  has  no  response. 

Defeasibility  is  pervasive  in  everyday  reasoning  but  is  not  handled  well  within  either 
formal  or  informal  logical  systems.  Logicians  tend  to  deal  with  defeasibility  by  tinkering  with 
the  premises  or  inference  rules  of  a  reasoning  system.  For  example,  they  might  add  the  falsity  of 
the  defeater  to  the  premises  in  the  argument— e.g.,  artillery  location  is  an  indicator  of  location  of 
attack  only  if  longer  range  artillery  has  not  been  developed  and  deployed.  The  problem  with  this 
tactic,  aside  from  computational  complexity,  is  that  it  blocks  reasoning  with  incomplete 
information.  The  falsity  of  all  possible  defeaters  would  have  to  be  positively  determined 
whenever  artillery  was  used  as  an  indicator  of  location  of  attack.  But  in  many  circumstances,  this 
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is  either  not  possible  or  not  worth  the  time.  As  the  conversation  between  MAJ  Kerr  and  MAJ 
House  continues,  more  exceptions  and  exceptions  to  exceptions  may  be  brought  forward.  Each 
new  addition  of  clauses  to  the  premises  would  ratchet  up  the  demand  for  information  before  the 
inference  can  be  regarded  as  valid.  As  a  result,  the  decision  maker  might  never  be  able  to  reach  a 
conclusion  at  all.  A  partial  solution  is  to  add  special  default  inference  rules,  so  that  the 
conclusion  follows  in  the  absence  of  positive  evidence  that  the  defeaters  are  true  (e.g.,  Reiter, 
1980).  Again,  however,  this  introduces  extreme  computational  complexity.  Another  problem  is 
that  the  list  of  potential  defeaters  is  indefinitely  long,  and  advance  specification  of  all  defeaters  in 
special  default  rules  may  be  impossible  even  in  principle.  The  set  of  defeaters  for  the  inference 
from  an  effect  to  a  cause,  for  example,  must  include  all  the  other  possible  causes.  Even  more 
importantly,  the  logical  approaches  provide  neither  guidance  nor  flexibility  in  determining  how 
long  the  process  of  generating  defeaters  and  collecting  information  about  them  should  go  on. 
Proficient  decision  makers  are  able  to  adapt  the  reasoning  process  to  specific  circumstances,  to 
act  decisively  on  a  subset  of  the  relevant  information  in  situations  where  that  is  necessary,  and  to 
demand  more  thinking  and  more  information  where  that  is  called  for. 

The  problem  of  defeasibility  invites  a  constructive  solution  involving  a  synthesis  of 
mental  model  theory,  dialogue  theory,  and  reliability.  Defeasibility  always  involves  an  initially 
incomplete  set  of  mental  models.  Put  the  other  way,  it  involves  the  discovery  of  possible  states  of 
affairs  that  were  not  previously  considered  but  which  are  relevant  to  the  conclusion  in  the  current 
context.  Thus,  it  lends  itself  to  a  semantic  mental  model-based  approach  that  represents  the 
alternative  possibilities  that  are  considered  in  reasoning  (Johnson-Laird  &  Byrne,  1991;  Johnson- 
Laird,  Legrenzi,  Girotto,  Legrenzi,  &  Cavemi,  1 999).  The  reasoning  process  itself  alternates 
steps  of  generating  new  possibilities  and  using  background  knowledge  or  explicit  inference  to 
evaluate  their  plausibility.  Dialogue  theory  provides  norms  for  the  process  of  challenge  and 
response  during  which  mental  models  are  elaborated  and  accepted  or  rejected.  Finally,  as  we 
shall  see,  judgments  of  reliability  determine  what  the  process  should  be  used  and  when  the 
process  should  stop  in  any  particular  situation 

As  the  critical  dialogue  progresses,  new  features  are  added  to  the  model,  either  to 
challenge  or  to  defend  the  proponent’s  conclusion.  Thus,  the  model  shows  how  argumentation 
expands  the  sharing  of  knowledge  between  dialogue  participants.  For  an  individual,  critical 
dialogue  has  a  function  of  eliciting  knowledge  that  may  not  otherwise  have  been  used  in  the 
current  problem.  The  features  elicited  in  critical  dialogue  are  represented  by  columns  in  the 
mental  model  tables.  Each  feature  is  a  dimension  along  which  possible  states  of  affairs  can  vary. 
Thus,  each  new  feature  increases  the  number  of  logically  possible  situations,  i.e.,  the 
combinations  of  truth  and  falsity.  For  example,  since  there  are  six  features  (columns)  in  the  final 
step  of  our  example,  there  are  actually  2x2x2x2x2x2  =  64  possible  states  of  affairs! 

Clearly,  it  would  be  impossible  for  humans  to  keep  that  many  possibilities  in  mind,  and 
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fortunately  it  is  not  necessary.  As  the  example  illustrates,  because  of  the  role  of  background 
beliefs  and  the  avoidance  of  explicit  deductive  inference,  the  actual  number  of  mental  modes  that 
needs  to  be  considered  is  much  lower,  and  does  not  necessarily  increase  much  at  all  as  new 
dimensions  are  introduced.  The  objective  of  the  proponent  is  to  reduce  the  number  of  mental 
models  until  all  the  survivors  contain  the  conclusion,  and  she  does  so  by  introducing  new 
considerations  that  interact  appropriately  with  background  knowledge.  In  addition,  concessions 
function  as  assumptions  which  reduce  the  range  of  alternatives  to  be  considered.  This  example 
required  explicit  representation  of  only  one  or  two  of  the  64  logically  possible  mental  models  at 
any  given  time. 

The  objective  of  the  opponent,  of  course,  is  to  increase  the  number  of  mental  models,  i.e., 
to  force  the  proponent  to  consider  and  respond  to  alternative  possibilities  in  which  her  conclusion 
is  not  the  case.  The  evolution  of  mental  models  for  both  the  proponent  and  opponent  thus 
provides  a  vivid  record  of  the  progress  of  a  critical  dialogue,  and  clarifies  the  kinds  of  moves  that 
each  side  should  make  in  order  to  persuade  the  other. 

RELIABILITY 

A  problem  that  is  not  addressed  by  either  mental  model  theory  or  dialogue  theory  is  the 
choice  of  a  strategy  that  will  reliably  achieve  external  objectives.  This  gap  exists  because  of  the 
internalist  character  of  both  mental  model  theory  and  dialogue  theory.  According  to  internalist 
theories,  criteria  for  assessing  the  acceptability  of  beliefs  must  always  refer  to  cognitively 
accessible  internal  representations,  and  not  external  facts  of  which  the  cognizer  was  not  aware. 
Dialogue  theory  refers  to  two  people  engaged  in  an  overt  verbal  exchange.  Despite  this  public 
character,  dialogue  theory  has  more  kinship  to  internal  approaches.  It  focuses  primarily  on 
internal  conformity  of  a  verbal  exchange  to  the  norms  of  a  particular  type  of  dialogue,  rather  than 
on  the  selection  of  the  dialogue  type  and  regulation  of  the  dialogue  itself  in  a  way  that  is 
appropriate  for  an  external  task.  Two  features  clinch  its  internal  status:  First,  the  norms  are 
applied  only  to  facts  that  are  known  to  one  or  both  of  the  participants.  Second,  the  evaluation 
focuses  on  proximal  or  internal  objectives  associated  with  a  particular  type  of  dialogue,  e.g., 
resolving  a  conflict  of  opinions,  rather  than  on  distal  or  external  objectives,  such  as 
accomplishment  of  a  task  or  mission.  Because  of  these  internal  norms  and  proximal  objectives, 
dialogue  theory  tends  to  describe  self-enclosed  games.  Its  internal  focus  is  responsible  for  the 
failure  of  dialogue  theory  to  adequately  address  three  key  issues:  The  selection  of  the  appropriate 
types  of  dialogue,  the  rules  for  bringing  a  dialogue  to  an  end,  and  how  to  determine  the  winner. 

All  of  these  issues  require  judgments  of  external  reliability. 

Dialogue  theory  does  not  address  the  rationale  for  choosing  a  particular  dialogue  type  on 
a  particular  occasion,  i.e.,  how  different  types  of  dialogues,  such  as  negotiation,  inquiry, 
persuasion,  information  seeking,  deliberation,  and  quarrel,  might  be  conducive  to  the 
accomplishment  of  different  real-world  objectives  (Walton,  1998).  The  same  dialogue  type  and 
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sequence  of  moves  might  be  judged  appropriate  in  one  context  but  not  in  another.  An  expert- 
consultation  dialogue,  with  appropriate  norms,  might  make  sense  when  one  participant  has 
significantly  more  knowledge  and  experience  than  the  other;  but  an  information  seeking 

dialogue,  with  different  norms,  should  be  used  when  one  party  merely  has  information  that  the 
other  party  lacks. 

Dialogue  theory  does  not  provide  an  adequate  solution  for  when  to  stop  a  dialogue.  For 
example,  in  the  critical  discussion  that  we  looked  at  above,  there  was  no  limit  to  the  number  of 
challenges  and  responses,  hence,  to  the  number  of  features  and  alternative  mental  models  that 
might  be  considered.  Participants  need  to  know  when  challenges  should  come  to  an  end  and  the 
current  best  conclusion  acted  upon,  and  this  usually  depends  on  external  context.  For  example, 
the  same  dialogue  might  justify  acceptance  of  a  conclusion  when  there  was  limited  time  or 
information  to  make  a  decision,  but  might  be  insufficient  to  justify  a  conclusion  when  more 
information  or  more  time  is  available  or  the  stakes  are  more  serious. 

Dialogue  theorists  address  the  issue  of  winning  and  loosing  in  terms  of  clear-cut  cases,  in 
which  either  the  proponent  retracts  her  original  assertion  or  the  opponent  withdraws  her 
challenge.  Real  cases  may  not  always  be  so  easy.  Time  constraints  may  bring  a  dialogue  to  an 
end  before  definitive  closure  is  achieved.  In  such  cases,  it  is  necessary  to  determine  which 
position  was  superior  at  the  time  the  dialogue  came  to  an  end,  taking  into  account  the 
opportunities  that  the  participants  had  to  challenge  one  another.  This  requires  judgments  about 
the  relative  reliability  of  different  belief  formation  processes  as  well  as  the  coherence  of  the 
alternative  mental  models  with  a  large  store  of  background  information. 

According  to  van  Eemeren  and  Grootendorst  (1992),  decisions  of  these  kinds  take  place 
during  the  opening  stage  and  the  concluding  stage  of  the  dialogue,  rather  than  during  the 
argumentation  stage.  For  example,  the  type  of  dialogue  should  be  agreed  upon  between  the 
participants  at  the  beginning  of  the  dialogue,  and  the  concluding  stage  determines  when  the 
dialogue  ends  and  who  won.  Segregating  them  into  different  stages  suggests  that  these  decisions 
are  qualitatively  different  from  argumentation  proper.  But  dialogue  theorists  do  not  address  how 
the  decisions  should  be  made.  Placing  them  in  different  temporal  stages  is  quite  artificial  and 
only  makes  matters  worse,  since  it  eliminates  the  possibility  of  continuous  review  of  the  dialogue 
based  on  new  information  acquired  during  argumentation.  Such  information  might  lead  to  a  shift 
from  one  type  of  dialogue  to  another  (Walton,  1998),  or  it  might  change  the  estimation  of  how 
the  risks  of  further  delay  balance  out  the  costs  of  an  incorrect  conclusion,  and  thus  affect  the 
decision  of  when  to  stop.  A  more  promising  direction  is  to  introduce  elements  of  externalist 
models,  which  take  into  account  likely  outcomes  and  their  associated  impact  on  objectives. 

To  help  dialogue  theory  bridge  the  gap  between  internal  and  external  concerns,  it  is 
necessary  to  provide  a  third  role,  that  of  a  judge,  in  addition  to  those  of  proponent  and  opponent 
(Figure  6).  All  three  of  the  issues  just  discussed  belong  among  the  duties  of  the  judge.  The  judge 
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evaluates  the  reliability  of  alternative  types  of  dialogues  for  the  current  context  and  purposes. 
The  judge  evaluates  the  status  of  the  argument  at  any  given  time  to  determine  the  most  plausible 
current  position,  i.e.,  the  winner  if  the  dialogue  were  to  end  at  that  moment.  And  finally,  the 
judge  continuously  weighs  the  value  of  continuing  a  particular  dialogue  versus  the  value  of 
stopping  and  committing  to  the  most  plausible  current  position. 


Control 


Figure  6:  Three  part  model  of  critical  thinking  in  terms  of  stages  and  roles  in  a  critical  dialogue. 

Figure  6  shows  that  each  component  of  our  critical  thinking  model  (Figure  1) 
corresponds  to  a  dialogue  theory  concept.  As  we  have  seen,  mental  models  correspond  to  the 
commitment  stores  of  proponent  and  opponent;  critical  dialogue  corresponds  to  the 
argumentation  between  proponent  and  opponent  in  which  the  mental  models  are  evaluated  and 
improved;  and  the  judge  determines  the  overall  reliability  of  the  process  and  regulates  it 
accordingly.  The  judge  is  subject  to  the  same  capacity  limitations  as  the  proponent  and  opponent, 
of  course,  and  will  not  generally  optimize  strategy  choices.  Rather,  in  accordance  with  the 
principles  of  bounded  rationality  (Simon,  1997;  Gigerenzer  &  Selten,  2001),  the  judge  will 
become  adapted  through  experiences  of  success  and  failure  in  the  use  of  various  cognitive 
processes  and  mechanisms  in  different  contexts.  The  judge  may  select  and  regulate  belief 
forming  strategies  based  on  relatively  automatic  processes  shaped  by  experience,  or  may 
evaluate  the  reliability  of  different  strategies  by  explicit  reasoning.  The  common  core  of  the 
judge  s  functionality  is  judgment  about  the  trustworthiness  of  a  cognitive  faculty  from  a 
standpoint  that  is  external  to  that  particular  faculty. 

The  introduction  of  a  reliability-based  judge  has  another  advantage.  It  generalizes  critical 
thinking  beyond  the  evaluation  of  explicit  reasoning  or  critical  dialogue.  Other  belief-generating 
faculties,  such  as  perception,  recall,  and  recognition  can  also  be  assessed  in  terms  of  their 
reliability,  even  though  they  do  not  themselves  involve  reason-giving  and  critiquing.  Thus,  we 
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can  think  of  the  judge  as  evaluating  not  only  the  reliability  of  different  dialogue  types,  but  more 
generally,  evaluating  the  effectiveness  and  efficiency  of  alternative  cognitive  faculties,  and 
decision  making  and  problem  solving  strategies.  In  some  situations,  taking  time  to  reason  may 
not  be  the  best  solution. 

CONCLUSIONS:  WHAT  ABOUT  CRITICAL  THINKING  IN  THE  ARMY? 

It  is  appropriate  now  to  summarize  some  of  the  implications  of  this  theory  for  the 
challenges  we  laid  down  at  the  beginning.  Here  again  are  some  of  the  potential  difficulties  of 
implementing  critical  thinking  training  in  the  Army  context: 

Is  Critical  Thinking  Consistent  with  Tactical  Battlefield  Constraints? 

■  Will  critical  thinking  on  the  battlefield  take  too  much  time?  Would  that  time  be  put  to 
better  use  gaining  a  jump  on  the  enemy? 

■  Will  critical  thinking  result  in  a  loss  of  the  confidence  necessary  for  decisive 
leadership  and  action?  Will  it  undermine  the  “will  to  fight”? 

The  external  layer  of  critical  thinking,  i.e.,  the  assessment  of  reliability,  is  the  source  of  a 
stopping  rule  for  the  process  of  challenging  and  response.  It  demands  that  the  critical  thinker  stay 
focused  on  real  task  objectives.  Reflective  reasoning  is  one  tool  among  others,  including 
recognitional  decision  making,  and  should  be  used  when  and  only  when  it  will  increase  the  odds 
of  success.  There  are,  however,  many  examples  in  which  a  little  time  spent  thinking  saved  much 
more  time  in  execution  (e.g.,  Cohen  &  Thompson,  2001).  Because  of  the  external  layer, 

however,  critical  thinking  never  involves  an  endless  exploration  of  alternative  possibilities  with 
no  end  in  sight. 

The  critical  dialogue  layer  of  critical  thinking  permits  a  variety  of  different  reasoning 
styles  that  differ  in  how  free-ranging  the  consideration  of  alternative  possibilities  may  be.  In  time 
stressed  situations,  a  more  constrained  reasoning  process,  in  which  basic  assumptions  are  not 
questioned,  leads  to  more  rapid  decision  making.  Explicit  recognition  of  the  mode  of  dialogue 
that  has  been  adopted  among  team  members  may  actually  speed  up  communication  and 
reasoning.  Confidence  is  typically  increased  by  a  disciplined  exploration  of  relevant  and 
significant  alternative  possibilities. 

Is  Critical  Thinking  Consistent  with  other  Battlefield  Skills? 

■  Will  critical  thinking  skills  trump  experience  or  leadership  qualities  on  the  battlefield, 
which  might  in  fact  lead  to  better  decisions? 

■  Will  critical  thinking  be  too  “critical”?  Will  it  stifle  innovation  or  the  development  of 
new  tactics  and  techniques? 
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The  external  layer  of  critical  thinking  involves  choosing  the  most  reliable  process  for  a 
given  decision.  For  experienced  leaders,  the  most  reliable  method  sometimes  involves  trust  in 
their  own  gut  feel  for  a  situation. 

As  far  as  innovation  goes,  the  dialogue  layer  of  critical  thinking  is  not  “critical”  in  a 
narrow  sense.  It  not  only  evaluates  possibilities,  it  generates  new  possibilities.  The  space  of 
alternatives  is  constantly  changing  as  a  result  of  the  challenge  and  response  process.  The 
construction  of  these  mental  models  does  not  necessarily  proceed  in  a  rigid  step  by  step  fashion. 
In  the  context  of  a  permissive  critical  dialogue,  any  assumptions  may  be  questioned  and 
retracted.  Alternative  mental  models  are  evaluated  in  terms  of  their  overall  coherence  with  a 
system  of  beliefs.  The  interconnectedness  of  beliefs  in  a  coherence-based  system  can  lead  to 
rapid,  creative  shifts  in  the  understanding  of  a  situation,  similar  to  the  paradigm  shifts  that  T. 
Kuhn  (1996)  describes.  Such  shifts  may  involve  the  simultaneous  modification  of  numerous 
assumptions,  beliefs,  and  plans. 

Is  Critical  Thinking  Appropriate  for  Military  Organizational  Structure? 

■  Will  critical  thinking  encourage  inappropriate  initiative?  Will  it  disrupt  the  chain  of 
command  and  degrade  coordination  and  synchronization  on  the  battlefield?  Put 
another  way,  is  the  Army  too  centralized  and  hierarchical  for  critical  thinking  to 
flourish? 

■  Will  critical  thinking  hinder  the  development  of  trust  in  diverse,  multi-cultural  teams 
because  it  is  “Western,  masculine,  individualistic,  adversarial,  and  coldly  rational” 
(Atkinson,  1 998,  p.  1 2 1 ). 

Critical  thinking  is  most  suited  to  situations  in  which  individuals  have  significant 
autonomy  and  responsibility,  and  such  situations  are  likely  to  increase  in  frequency  in  future 
Army  missions.  But  critical  thinking  can  function  at  many  different  levels,  e.g.,  in  the 
performance  of  virtually  any  non-routine  task.  The  dialogue  layer  provides  a  series  of  dialogue 
types  that  vary  in  the  extent  to  which  assumptions  are  questioned.  The  higher  the  level  of 
initiative,  the  more  far-reaching  the  exploration  of  alternatives  might  be.  But  critical  thinking  at 
some  level  is  nearly  always  appropriate. 

As  for  cultural  diversity,  the  dialogue  layer  provides  a  framework  for  classifying  different 
styles  of  interaction.  This  framework  may  lead  to  more  stable  and  better  calibrated  expectations 
among  individuals  from  diverse  cultural  backgrounds.  It  also  allows  for  the  evolution  of  new 
styles  of  dialogue  that  may  be  better  suited  to  a  specific  team  or  context. 

Will  Critical  Thinking  Fit  into  Army  Training? 

■  Are  there  right  answers  in  critical  thinking?  If  so,  isn’t  this  just  a  new  phrase  for 
teaching  doctrine  and  tactics,  which  we  already  do?  If  not,  what  good  are  skills  that 
can’t  be  evaluated?  How  can  we  know  they  will  improve  performance? 
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■  Will  critical  thinking  instruction  consume  too  much  training  time?  How  will  we 
persuade  instructors  to  provide  that  time?  Does  critical  thinking  require  technical 
training  in  logic  or  decision  theory?  Does  it  require  stand-alone  courses?  How  will  we 
persuade  students  to  devote  their  time  to  the  study  of  critical  thinking? 

Metrics  for  critical  thinking  performance  focus  on  process  rather  than  product.  Both  the 
dialogue  layer  and  the  reliability  layer  evaluate  belief  acceptance  in  terms  of  the  processes  that 
led  to  it,  and  each  provides  relatively  unambiguous  evaluative  criteria.  Metrics  for  a  successful 
dialogue  measure  the  degree  to  which  an  actual  conversational  exchange  corresponds  to  the 
profile  of  the  relevant  type  of  dialogue.  For  example,  was  disagreement  acknowledged?  Were 
challenges  sought  out?  Were  they  answered?  Metrics  for  reliability  include  the  probability  that 
the  selected  cognitive  faculty  or  communicative  process  will  support  the  objectives  of  the  task 
under  the  prevailing  conditions.  For  either  dialogue  or  reliability  based  measures,  a  decision  may 
be  good  even  though  the  outcome  happens  to  be  bad,  and  conversely,  a  decision  may  be  bad 
even  though  there  was  a  lucky  outcome. 

Each  layer  of  critical  thinking  is  associated  with  a  specific  set  of  skills  and  training 
objectives.  For  example,  the  innermost,  mental  model  layer  involves  the  ability  to  generate 
possibilities  based  on  existing  elements,  the  ability  to  add  dimensions  to  the  space  of  situations, 
and  the  ability  to  evaluate  and  compare  mental  models  in  terms  of  their  internal  coherence  and 
compatibility  with  background  knowledge.  The  dialogue  layer  involves  awareness  of  different 
types  of  dialogues  with  different  rules  for  identifying  conflicting  positions,  for  challenging  and 
retracting  assumptions,  and  for  “winning”  and  “loosing.”  The  outermost,  reliability  layer  requires 
an  awareness  of  strengths  and  weaknesses  of  different  cognitive  processes  or  faculties,  and  the 
ability  to  make  appropriate  choices  based  on  the  circumstances,  e.g,  between  recognitional 
decision  making,  creative  brainstorming,  or  reflective  reasoning. 

Critical  thinking  skills  are  best  acquired  in  the  context  of  actual  decision  making.  Thus, 
critical  thinking  training  may  be  incorporated  relatively  seamlessly  into  subject  matter 
coursework,  exercises,  and  field  training.  Students  may  be  taught  through  coaching,  hints, 
feedback,  and  example,  in  addition  to  explicit  instruction  (see  Cohen,  at  al.,  2000a).  Critical 
thinking  training  can  also  be  given  as  a  standalone  course,  as  long  as  concrete  exercises  (e.g, 
tactical  decision  games)  are  emphasized.  None  of  the  relevant  skills  requires  specialized  training 
in  formal  logic,  decision  theory,  or  philosophy.  Nevertheless,  these  are  skills  that  need  some 
explicit  attention,  and  thus  it  would  be  best  for  instructors  to  receive  some  specialized  training.  A 

useful  first  step  might  be  the  development  of  a  brief,  intensified  critical  thinking  course  for 
instructors. 
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What  is  a  Team? 


Dynamic,  interdependent,  and  adaptive  interaction 
Common  goal,  mission,  or  objective 
Organizational  structure  of  team  members 
Each  individual  has  specific  tasks  or  functions 

Task  completion  requires  dynamic  communication  of 
information  and  coordination  of  task  activities 


(Salas,  Dickinson,  Converse  &  Tannenbaum,  1992) 
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Interdependence  of 
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I  Extent  of  common  goals 
I  Extent  of  task  interdependence 
I  Nature  of  the  interdependence 

-  time  (sequential,  parallel) 

-  outcome 

I  Precision  of  interdependence  in  space  and 
time 

Extent  of  task  specialization/overlap 
(possibility  of  backup) 
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What  Do  Teams  Do? 
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Team  Competencies  (i/3>: 

What  Do  We  Know  About 
Team  Knowledge? 

I  Task-specific  roles  and  responsibilities 
I  Knowledge  of  team  mission  and  objectives 
I  Knowledge  of  team  norms  and  resources 
Knowledge  of  team  “state” 

Teammate  characteristics  (behavior  adjustment) 
Shared  task  models  j 
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What  Do  We  Know  About 
Team  Skills? 

Conflict  resolution 
Task  assertiveness 
Leadership 

Supporting/back-up  behavior 
Mutual  performance  monitoring 
Coordination  &  communication 
Adaptability:  behavioral  discretion 
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What  Do  We  Know  About 
Team  Attitudes? 


■  Collective  orientation 

■  Mutual  trust 

■  Importance  of  teamwork 

■  Team  cohesion 

■  Shared  vision 

■  Collective  efficacy 

■  Team  orientation  (morale) 
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Key  Things  We  Learned 
About  Command  Teams 

''Good  individual  skills  are  necessary,  but  not 
sufficient,  to  high  team  performance 

^Teamwork  skills  are  distinct  from  taskwork  skills 

In  high-performing  teams,  team  members: 

-  Anticipate  each  other’s  needs 

-  Practice  pro-active  back-up  behaviors 

-  Have  a  dynamic  shared  “mental  model”  of  the  task 
and  the  team 

Provide  continuous  mutual  feedback  on  performance 

“  Adapt,  adapt,  adapt... 

Question:  flow  would  critical  thinking 
enhance  or  support  these  behaviors? 
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To  Enhance  Team  Performance 

Change  team  membership 
Modify  team  tasks,  workflow,  structure 
Improve  team  interactions/processes 
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Enhance  individual  team  member  capabilities 
Provide  support,  resources,  performance  aids 
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A  Shared  Mental  Model  is  a  necessary,  but  not  a  sufficient, 
requirement  for  superior  team  performance  (need  teamwork  skills) 
Implications  for  TACT  Team  Training 

-  Implicit-Explicit  Adaptive  Coordination 

-  Leader’s  Situational  Update 
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TACT  Training  Model 

I  Train  team  members  to  recognize 

-  Stressors  in  the  tactical  environment 

-  Signs  of  stress  in  the  team 

Teach  the  team  alternative  communication  and 

coordination  strategies 

Train  the  team  to  match  each  coordination  strategy  to 
conditions  of  the  team  environment 

Practice  and  feedback  this  adaptive  behavior  at  the 
team  level 

Add  leader’s  situation  updates  to  periodically  calibrate 
the  team 
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Team  Performance  Measures 


Core  Elements  of  Measurement  Approach 

Why? 

What? 

Driven  by  targeted  questions: 

Measure  both  outcome  and  process: 

.  Team  organization  and  distribution 
of  roles  and  responsibilities 
.  Impacts  of  new  technologies  and 
decision  support 
.  Team  communication  and 
coordination 

.  "Make  the  right  decisions" 

.  "Make  the  decisions  right" 
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Metrics  of  Team  Communication  &  Coordination 


Measures  of  Flow  of 
information  &  Communication 
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■  Type  of  Communication 
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Messages 


Second  Order  Metrics  of  Information 
Management  and  Coordination 

■  Information  compression  ratio 

-  Relationship  between  incoming  and 
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■  Quality  ratio 

-  Quality  of  information  filters 

■  Anticipation  ratio 

-  Measures  staffs  ability  to  anticipate 

information  needs  of  others 
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■  Pro-action  ratio 
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Team  Critical  Thinking 


ENGINEERING 


What  is  Team  Critical  Thinking? 

Control  process  by  which  teams: 

-  Observe  their  own  performance, 

-  Reason  about  themselves  and 

Consequently  act  to  modify  the  team  processes  that  produce 
their  performance 


Team  Processes 


Team 


Performance 


Action 


to  Change 


Reasonin 


Observation 
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Three  Key  Decisions  Specific  to 
Critical  Thinking  in  Teams 

Decision  to  Communicate 
Decision  to  Coordinate 
Decision  to  Reconfigure 
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he  Decision  to  Communicate 

I  Communications  dual  purpose: 

-  Transfer  of  information 

-  Update  of  each  other’s  mental  models 

I  Where  is  the  source  of  the  info  I  need? 

I  Who  could  benefit  from  the  info  I  have? 

I  Key  concepts: 

-  Organizational  knowledge 

-  Information  state  awareness 

-  Conflict  resolution 

—  ;  .  ■  . . .  t 

How  can  critical  thinking  help  align  team  members’ 

understanding  of  their  information,  tasks,  and  goals 
through  proper  team  communication? 
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The  Decision  to  Coordinate 

'  Coordination:  Management  of  overlap 
among  team  members 

Four  Dimensions:  Goals,  Resources, 
Information,  and  Tasks 

Key  Concepts: 

-  Implicit  Coordination 

-  Anticipation 

-  Workload  distribution 


Is  critical  thinking  the  principal  ingredient 
for  team  adaptive  coordination  ? 
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Team  Action  Coordination 
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Based  on  Observation 

Based  on  Communication  l 

■  Depends  on 

■  Depends  on 

-  amount  of  visual  and 

-  availability  of  1 
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available  about 

the  situation 

teammates  (co- 

—  different  pictures  j 

located  versus 

among  teammates 

distributed  teams) 

require  push/pull  of  j 

information 

-  communication 

-  need  for  team  leader 

channels  available 

to  direct  action 
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-  need  for  backup 

-  availability  of  j-* 
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Example:  Team  Workload  & 


Awareness  Across  Team  Members 


•  Average  workload 
increases  30%  as 
operation  progresses 

•  Pre-training:  Large 
discrepancy  between 
self-  and  perceived 
workload  by  other  cells 

•  Post-training:  More 
accurate  mutual 
perception  of  workload 


Common  Operating  Picture 
Calibrated  Team 


PhascI  Delta  =  41% 


Phase  n  Delta  _  go/0 
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The  Decision  to  Reconfigure 

Outer  Loop  of  Adaptation 
Detecting  the  cues  and  needs  for  change 
Understanding  the  paths  to  new  structures 
Key  concepts: 

-  Congruence 

-  Organizational  inertia 

-  Organizational  learning 

-  Dynamic  reconfiguration 


Can  team  structure  be  designed  to  support 
the  practice  of  team  critical  thinking? 
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Summary 


I  Significant  performance  and  training  research 
for  military  command  teams 

■  Very  little  research  in  critical  thinking  in  teams 

I  Multiple  team  roles  and  team  structures  render 
the  problem  of  team  CT  training  more  complex 

-  Distributed  teams 

-  Virtual  teams 

-  Ad  hoc  teams 

-  Vertical  and  horizontal  structures 

. . .  .  ^ 

Complexity  and  diversity  of  military  missions  demand 
adaptive  teams  of  warfighters  capable  of 
- thinking  about  the  wav  they  think  as  teams  88 
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A  Simulation  Tool  for  Critical  Thinking  Training1 

Marvin  S.  Cohen 
Cognitive  Technologies,  Inc. 

COMPUTATIONAL  LIMITS  AND  ATTENTION  SHIFTING 

The  development  of  training  system  technology  that  we  will  discuss  here  is  part  of  a 
larger  research  effort,  in  which  we  are  trying  to  understand  and  simulate  the  structure  and 
dynamics  of  human  cognition  and  decision  making  processes.  In  doing  this,  we  are  exploring  the 
linkages  among  several  fields  and  building  new  models  that  are  informed  by  data  from  each  of 
these  fields.  Our  aim  has  been  to  integrate  high-level  cognitive  modeling,  connectionist 

knowledge  representation,  and  methods  from  optimal  control  (specifically,  approximate  dynamic 
programming). 

This  work  represents  a  synthesis  of  the  Recognition/Metacognition  theory  (Cohen  et  al., 
1996,  1998,  2000a,  2001)  with  work  on  computer-based  inference  within  the  Shruti  system,  by 
Professor  Lokendra  Shastri,  of  the  Computer  Science  Department,  University  of  California  at 
Berkeley  (Shastri,  1992,  199a,  199b;  Shastri  etal.,  1993,  1996,  1997,  1998).  The  immediate  goal 
of  this  work  was  to  develop  a  tool  that  can  perform  rapid  recognitional  inferences  and  planning 
within  a  large  (expert)  belief  network,  exemplify  human  limitations  on  computational  resources 
and  attention;  and  implement  metacognitive  control  process  that  regulate  recognitional 
processing,  help  overcome  computational  limitations,  and  deal  with  uncertainty.  Such  a  tool 
could  form  the  basis,  in  subsequent  research,  for  the  development  of  an  adaptive  training  system. 

According  to  the  Recognition/Metacognition  theory,  decision  makers  structure  complex 
and  voluminous  knowledge  about  their  world  into  causal  models  that  enable  them  to  rapidly 
generate  coherent  interpretations  and  plans  in  response  to  an  influx  of  new  evidence  and 
observations.  We  model  these  rapid  recognitional  processes  using  Shruti,  a  connectionist 
architecture  for  reflexive  inference.  Critical  limits  on  dynamic  access  to  long  term  memory 
(LTM)  emerge  naturally  from  the  computational  structure  of  Shruti  and  the  neuro-biological 
constraints  it  respects.  These  limits  effectively  insist  that  not  all  information  known  by  the  agent 
can  be  brought  to  bear  at  the  same  time.  One  of  the  key  differences  between  experts  and  novices 
is  in  how  they  structure  knowledge  to  manage  these  resource  limitations  and  apply  the 
appropriate  information  during  reasoning. 


This  work  was  partially  funded  by  Contract  No.  DASW01-97-C-0038  with  the  Army  Research  Institute.  Dr. 
Sharon  Riedel  was  the  technical  officer.  The  research  was  also  independently  funded  by  the  Office  of  Naval 

Research  (Contracts  N00014-95-C-0182  and  N00014-00-M-0070)  and  the  National  Science  Foundation 
(Contract  DMC986141 1). 
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The  existence  of  such  limits  means  that  inference  and  planning  processes  must  be  capable 
of  dynamically  determining  the  scope  of  active  human  memory  from  which  they  draw  at  any 
given  time,  and  of  remaining  coherent  within  those  limits.  These  changes  of  scope  underlie  the 
fluidity  with  which  a  reasoner  is  able  to  focus  limited  computational  resources  at  different  levels 
of  spatial  and  temporal  abstraction  (the  chunking  problem  in  AI),  and  extend  planning  horizons 
from  moments  to  years  and  back  to  moments.  At  the  same  time,  this  need  for  fluid  changes  in 
focus  introduces  the  necessity  for  an  adaptive  dynamics  of  executive  attention.  The  mechanisms 
of  attention  shifting,  in  turn,  form  a  developmental  basis  for  acquiring  skilled  metacognitive 
behaviors,  which  monitor  and  regulate  recognitional  processing. 

According  to  the  model  described  by  Cohen  et  al.  (2000a),  metacognitive  processes  guide 
the  focus  of  attention  within  active  memory.  Studies  by  the  project  team  and  others  suggest  that 
these  metacognitive  processes  include  monitoring  recognitional  results  for  different  kinds  of 
uncertainty,  including  gaps  in  knowledge,  conflicting  evidence  or  goals,  and  unreliable 
assumptions;  attempting  to  fill  gaps,  resolve  conflicts,  and  evaluate  assumptions,  e.g.,  by 
generating  and  considering  alternative  hypotheses  to  explain  evidence  and  alternative  plans  to 
achieve  goals;  and  regulation  of  the  time  taken  for  reflection  versus  immediate  action,  based  on 
the  costs  of  delay,  the  stakes,  and  the  degree  of  uncertainty.  Our  computational  research  suggests 
that  these  metacognitive  processes  may  develop  naturally  as  extensions  of  skilled  attention- 
shifting  behaviors  within  a  resource-limited  active  memory.  Metacognition  enables  reasoning 
about  highly  mediated  relationships  in  long-term  memory,  i.e.,  interdependencies  that  are 
implicit  in  long-term  memory,  but  which  are  too  distant  to  combine  to  influence  decisions. 
Metacognitive  processes  thus  forge  more  distant  connections  and  introduce  a  wider  perspective 
within  a  computationally  constrained  reasoning  process. 

We  use  Shruti  to  model  both  rapid  reflexive  processes  and  the  reflective  processes  that 
monitor  and  regulate  them.  In  this  way,  the  resource  limitations  that  Shruti  implies  are  shared 
across  reflexive  and  reflective  processing.  Therefore,  a  reflective  decision  maker  achieves  less  in 
any  particular  cycle  of  reflexive  processing,  but  may  receive  a  net  benefit  by  extending  the  span 
of  reflexive  processing  across  multiple  cycles  of  attention  shifting. 

APPLICATION  FOR  TRAINING 

The  great  promise  of  computer-assisted  training  is  its  potential  to  track  the  progress  of 
individual  students  in  real  time.  Feedback  and  training  content  might  be  adapted  to  individual 
students  on  at  least  three  different  levels:  (1)  at  the  lowest  frequency,  to  enduring  personal 
cognitive  styles  and  overall  goals;  (2)  at  an  intermediate  rate  of  change,  to  current  level  of 
ability;  and  (3)  at  the  most  transient,  high-frequency  level,  to  the  momentary  state  of  strategy 
execution,  fatigue,  attention,  or  stress.  A  key  technical  hurdle  at  all  of  these  levels  is  flexibility. 
Once  the  knowledge  base  for  a  particular  problem  situation  has  been  coded,  the  feedback  tools 
and  adaptation  policies  should  be  able  to  recognize  and  evaluate  a  range  of  unacceptable  and 
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acceptable  variations  in  student  responses,  at  various  levels  of  abstraction,  and  over  long  and 
short  time  periods,  without  requiring  that  all  variations  and  their  significance  be  explicitly 
anticipated  by  training  designers  and  scripted  in  advance  in  the  training  system.  Flexible, 
adaptive  training  of  this  kind  requires  an  advanced  computer-based  model  of  the  targeted 
decision  making  skills,  going  beyond  procedural  rules  for  predicting  and/or  tracking  molecular 
responses.  A  bonus  of  this  kind  of  flexibility  will  be  its  extensibility  to  new  exercises  within  the 
same  situation  and  to  new  situations  in  the  same  domain. 

In  the  rest  of  this  paper,  we  provide  an  overview  of  some  of  the  distinctive  features  of  the 
simulation  tool,  on  both  the  recognitional  (reflexive)  and  metacognitive  (reflective)  sides,  and 
indicate  how  they  support  one  another’s  functioning.  We  will  illustrate  how  they  work  in  a 
simple  example.  We  will  also  briefly  outline  how  these  features  lend  themselves  readily  to 
visualization  within  a  graphical  user  interface.  A  more  detailed  account  of  the  system  can  be 
found  in  Cohen  et  al.  (2000c). 

REFLEXIVE  AND  REFLECTIVE  PROCESSING:  AN  EXAMPLE 

In  the  Sanna  s  Post  scenario,  you  are  the  commander  of  a  reinforced  rifle  company 
(which  we  shall  call  Company  A),  whose  mission  is  to  provide  flank  security  for -the  other 
companies  in  the  battalion  and  to  be  prepared  to  become  the  main  effort  if  necessary.  At  present, 
the  other  companies  are  heavily  engaged  to  the  east,  supporting  the  brigade  fight,  and  your 
company  must  prevent  reinforcement  by  the  enemy  from  the  west.  You  receive  a  scout  report 
that  enemy  supply  vehicles  and  fuel  trucks,  as  well  as  some  armored  vehicles,  are  in  the  vicinity 
of  the  town  of  Sanna's  Post.  There  is  a  road  through  Sanna’s  Post  that  runs  eastward  to  a  ford 
over  the  Modder  River,  and  from  there  to  the  area  of  the  battalion  fight  (Figure  1)  What  do  you 
do? 
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FIGURE  1:  Sanna’s  Post  scenario  map.2 

Figure  2  represents  a  possible  initial  recognitional  response  to  this  situation.  It  will  help 
us  introduce  some  important  basic  features  of  Shruti’s  model  of  recognitional  processing. 

Figure  3  and  Figure  4  represent  the  results  of  shifting  attention  under  the  control  of  the  reflective 
system,  and  will  help  us  discuss  how  reflective,  metacognitive  processes  monitor  and  regulate 
recognition. 


MacIntyre,  Capt  Douglas  J.  “Tactical  Decision  Game  #97-4:  Battle  of  Sanna’s  Post.”  Marine  Corps  Gazette  April 
1997.  Quoted  with  permission  by  Steve  M.  Crittenden,  Managing  Editor,  Marine  Corps  Gazette ,  Box  1775 
Quantico,  VA  22134, 4  Feb  99. 
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perceptual  / 
linguistic  input 


figure  2:  Initial  recogmtional  response  to  perceptual  input  in  the  Sarnia's  Post  example.  Attention  is 
tocused  by  perceptual  inputs  on  supply  trucks  at  Sanna’s  Post  (shaded  box).  The  circle  represents  the 
spread  of  activation,  and  labels  Al,  A2,  and  A3  represent  the  order  in  which  nodes  receive  activation. 
Intent  mental  model  components  are  also  labeled  (purpose ,  intent,  opportunity,  capability,  action). 


BASIC  FEATURES  OF  THE  REFLEXIVE  SYSTEM 

Shruti  enables  a  connectionist  representation  of  causal  mental  models.  In  Figure  2, 
the  perception  of  trucks  at  Sanna’s  Post  activates  knowledge  in  the  commander’s  long-term 
memory.  Some  of  this  knowledge  is  in  the  form  of  an  enemy  intent  mental  model  with  four 
active  components:  purpose,  opportunity,  intent,  and  actions  (Cohen  et  al.,  2000a).  The  long¬ 
term  memory  of  the  commander  includes  a  belief  that  the  enemy's  purpose  is  to  reinforce  the 
fight  against  the  battalion,  and  that  a  likely  opportunity  to  do  this  is  along  the  road  through 
Sanna’s  Post.  For  these  reasons,  the  enemy  intends  to  use  Sanna’s  Post  as  a  logistics  base,  and 
the  supply  vehicles  indicate  that  the  enemy  has  already  taken  action  to  implement  the  intent. 
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Additional  actions  that  might  be  expected  based  on  this  intent  are  use  of  the  road  by  enemy 
armored  columns  heading  east. 

Weights  in  the  reflexive  system  can  be  adapted  to  the  statistical  properties  of  the 
environment  through  experience.  The  Shruti  simulator  tunes  network  weights  and  rule-strengths 
via  supervised  learning,  using  a  form  of  backpropagation.  These  weights  reflect  the  co¬ 
occurrences  of  concepts  that  define  mental  models. 

Reasoning  proceeds  both  backward,  to  find  explanations,  and  forward,  to  generate 
predictions.  For  example,  the  perception  of  trucks  at  Sanna’s  Post  activates  explanatory  beliefs 
regarding  intent,  purpose,  and  opportunity.  These  in  turn  activate  a  prediction  of  future  events, 
i.e.,  the  appearance  of  armored  vehicles  on  the  road.  Combination  rules  in  Shruti  capture 
predictive  reasoning  and  abductive  reasoning,  as  well  as  taxonomic/semantic  reasoning.  As  a 
result  of  the  latter,  rules  framed  in  terms  of  general  categories  can  be  activated  by  information 
about  instances  of  those  categories. 

The  system  propagates  values  as  well  as  belief,  and  settles  on  actions  at  the  same 
time  as  it  settles  on  a  situation  interpretation.  Changes  in  belief  lead  to  the  activation  of  goals, 
and  the  activation  of  goals  influences  the  direction  of  attention  to  other  beliefs,  and  ultimately  the 
release  of  action.  As  we  have  seen  in  Figure  2,  the  observation  of  trucks  leads  to  a  prediction 
(i.e.,  enemy  use  of  the  road  to  reinforce  the  battalion  fight),  which  has  negative  expected  utility. 
The  utility  of  that  predicted  event  changes  the  salience  of  beliefs;  in  particular,  it  heightens  the 
degree  and  persistence  of  activation  of  all  the  causes  of  the  event,  including  enemy  intent, 
purpose,  and  opportunity.  Thus,  belief  propagated  from  an  event  to  its  causes  is  heightened  by 
the  (positive  or  negative)  utility  of  the  event.  In  parallel,  utility  itself  propagates  to  causally 
related  events  over  which  the  decision  maker  might  have  some  control.  For  example,  the 
negative  expected  utility  of  enemy  reinforcements  on  the  road  leads  to  activation  of  positive 
expected  utility  for  any  friendly  action  that  can  prevent  it  from  being  carried  out.  One  such 
action  is  destroying  the  enemy  in  Sanna’s  Post.  Metacognitive  processes,  as  we  shall  see,  are 
likely  to  shift  attention  to  this  possible  action. 

The  reflexive  system  uses  dynamic  variable  binding  to  keep  track  of  objects  and  the 
roles  they  play  in  relations.  This  feature  is  virtually  unique  among  rapid,  parallel  systems.  In 
this  example,  it  enables  the  system  to  know  that  the  same  entities  (i.e.,  Sanna’s  Post,  the  road,  the 
enemy)  recur  in  different  parts  of  the  inference,  i.e.,  are  bound  by  different  predicate  nodes  of  the 
inference  network.  These  identities  are  necessary  for  the  validity  of  the  conclusions.’  Traditional 
models  of  associative  processing  support  “associations  of  ideas,”  but  do  not  support  the  specific, 
relational  inferences  that  people  quickly  and  accurately  arrive  at.  For  example,  without  object 
tracking  and  enforcement  of  identity  constraints  by  rules,  it  would  be  possible  to  conclude  that 


to  ShmU  object  identity  is  represented  by  temporal  synchrony  of  firing  at  nodes  in  different  places  in  the  network 
and  identity  constraints  on  the  application  of  rules  are  enforced  by  temporal  pattern  matching. 
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because  trucks  were  observed  at  Same’s  Post,  the  enemy  intended  to  use  Fontain  as  a  logistics 
post.  It  might  be  possible  to  infer  that  because  this  force  intended  to  use  Sanna’s  Post  for 
logistics,  that  another  force  intended  to  use  a  different  local  road  to  move  armored  vehicles. 

The  reflexive  system  uses  parallel  processing  to  achieve  scalability  and  speed. 

Human  recognitional  reasoning  is  extremely  rapid  over  a  very  large  knowledge  base,  on  the 
order  of  500  milliseconds  or  less.  The  parallel  character  of  reflexive  processing  is  illustrated 
Figure  2.  The  node  labeled  A I  is  the  first  to  be  activated,  by  perceptual  inputs,  and  A2  is 
activated  next.  However,  the  four  nodes  labeled  A3  are  all  activated  simultaneously,  because 
they  are  all  two  layers  removed  from  the  initial  activation.  Because  computations  at  both  the 

reflexive  and  reflective  levels  are  parallel,  time  increases  as  a  linear  function  of  the  size  of  the 
network/ 

Predictions  regarding  resource  constraints  derive  naturally  from  the  representational 
and  computational  features  of  the  system.  These  features  account  for  limits  on  the  amount  of 
long-term  memory  that  can  be  active  in  working  memory  at  one  time.  In  Figure  2,  for  example, 
activation  beginning  at  A1  spreads  to  two  additional  layers  (A2  and  A3),  but  no  further/  In  order 
to  bring  more  knowledge  into  play,  the  commander  will  have  to  shift  attention. 

Shruti  acquires  and  stores  aggregated  information  about  what  lies  beyond  the 
current  edge  of  the  active  network.  Whenever  a  node  is  on  the  edge  of  the  currently  active 
network  (i.e.,  the  A3  nodes  in  Figure  2),  aggregated  information  stored  at  that  node  comes  into 
play,  representing  the  average  historical  effects  of  currently  inactive  information  that  is  linked  to 
that  node  in  long-term  memory.  Instances  of  aggregated  historical  information  are  labeled  as 
assumptions,  because  the  validity  of  inferences  within  the  active  part  of  the  network  depends  on 
(at  least  implicitly)  assuming  that  the  aggregated,  historical  information  in  fact  fits  the  present 
situation.  Acting  on  such  average  information  is  a  big  part  of  what  meant  by  “acting  on  habit,” 
i.e.,  failing  to  remain  mindful  of  the  particulars  of  the  situation.  In  this  model,  assumptions  of 
this  sort  are  a  principal  target  of  reflective,  metacognitive  monitoring. 


In  add. non  to  versions  of  Shrut.  that  run  on  serial  platforms,  a  version  of  Shruti  has  also  been  implemented  on  a 
paraHe!  machine -  the s  CM-5,  (Mani,  1995;  Shastri  &  Mani,  1997).  The  resulting  system  can  encode  knowledge  bases 
with  more  than  500  000  (randomly  generated)  rules  and  facts,  and  yet  respond  to  a  range  of  queries  requiring 
envanons  of  depth  five  m  under  250  milliseconds.  Even  queries  with  derivation  depths  of  eight  are  answered  in 
well  under  a  second.  We  are  actively  exploring  the  possibility  of  mapping  Shruti  onto  a  more  readily  available 
para  e  processor,  such  as  a  network  of  workstations  (NOW),  Beowulf  cluster,  or  array  of  StrongARM  processors. 

ara  el  processor  solutions  for  Shruti  have  been  studied  extensively  in  the  context  of  the  CM-5.  The  results  of  that 
research  should  apply  especially  well  to  a  StrongARM  array,  which  shares  the  extremely  low  message  latency  of  the 

Because  Shruti  uses  temporal  synchrony  for  object  identify,  finite  bandwidth  means  that  (i)  only  a  limited  number 
of  objects  can  be  tracked  at  any  given  time,  and  (ii)  ifjitter  increases  with  the  length  a  signal  travels  in  the  lono-term 
memory  network,  accurate  inference  about  objects  (as  opposed  to  coarser  associations  of  ideas)  is  limited  in  depth 
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Three  kinds  of  aggregated  information,  or  assumptions,  are  shown  in  Figure  2: 

■  Prior  probability  of  a  causal  explanation  (e.g.,  purpose  &  opportunity).  Two  of  the 
A3  nodes  in  Figure  2  are  possible  causes  of  the  trucks’  presence  at  Sanna’s  Post,  e.g., 
enemy  purpose  and  enemy  opportunity.  The  observation  of  trucks  provides  some 
support  for  beliefs  about  these  causes,  specifically,  that  the  enemy’s  purpose  is  to 
reinforce  the  fight  and  that  the  road  through  Sanna’s  Post  is  a  likely  opportunity  for 
doing  so.  To  infer  the  new  strength  of  these  beliefs  after  observing  the  trucks,  the 
impact  of  that  evidence  must  be  combined  with  the  prior  degree  of  belief  in  those 
causes.  This  prior  belief  is  based  on  the  past  frequency  with  which  an  enemy  of  this 
type  had  a  purpose  (or  opportunity)  of  this  type.  This  aggregated  information  does  not 
take  into  account  specific  features  of  the  current  situation,  which  might  make  it 
different  from  the  historical  average.  For  example,  there  may  be  further  causal  links 
that  suggest  that  this  will  (or  will  not)  be  the  enemy’s  purpose,  e.g.,  aspects  of  enemy 
doctrine  or  the  historical  practice  of  the  enemy  commander,.  Nodes  representing  these 
specific  possibilities,  even  if  they  do  exist  in  long-term  memory,  are  not  currently  in 

working  memory,”  i.e.,  they  have  not  been  activated  in  the  current  network. 

■  Expected  utility  of  an  event  ( e.g.,  moving  armored  reinforcements  along  the  road 
through  Sanna’s  Post).  One  of  the  A3  nodes  in  Figure  2  is  the  predicted  event, 
moving  reinforcements  along  the  road  through  Sanna’s  Post.  Much  of  the  negative 
utility  of  this  event  is  indirect,  inherited  from  its  historical  association  with  further 
events  (to  which  it  causally  contributes)  that  are  more  directly  undesirable:  i.e.,  an 
increase  in  friendly  casualties  and  a  higher  chance  of  the  enemy’s  prevailing.  Over 
past  experience,  the  positive  and  negative  utility  from  these  and  other  subsequent 
events  has  propagated  back  to  the  precursor  events.  As  a  result,  aggregated 
information  about  the  expected  utility  of  moving  armored  reinforcements  toward  the 
fight  is  stored  at  this  event  node.  However,  as  with  prior  probabilities,  this 
information  does  not  take  into  account  the  specifics  of  the  present  situation.  For 
example,  it  might  even  be  desirable  for  the  enemy  to  try  to  move  reinforcements  on 
that  road  in  this  situation,  if  expected  rains  are  likely  to  make  the  road  impassable. 

This  information,  if  it  exists  in  long  term  memory  at  all,  is  not  currently  active. 

■  Feasibility  of  an  action  (attacking  Sanna ’s  Post).  Any  event  is  a  potential  action  or 
goal,  when  it  is  actually  within  a  decision  maker’s  power  to  bring  about  or  prevent  the 
event.  Thus,  expected  utility  propagates  from  a  predicted  event  back  to  other  events 
that  can  causally  affect  it  and  over  which  the  decision  maker  might  have  some 
control.  Positive  or  negative  expected  utility  (unlike  belief)  is  propagated  to  an  event 
only  to  the  degree  that  action  to  influence  the  occurrence  of  the  event  is  feasible. 
Feasibility  information  is  stored  at  an  event  node  and  consists  of  aggregated 
information  about  the  results  of  trying  to  accomplish  or  prevent  that  event  in  the  past. 
For  example,  in  Figure  2,  the  enemy  s  intent  to  use  Sanna’s  Post  for  logistics  receives 
negative  expected  utility  if  it  is  feasible  to  prevent  it,  e.g.,  by  destroying  the  enemy  at 
Sanna  s  Post.  Since  it  is  on  the  edge  of  the  active  network,  destroying  the  enemy  at 
Sanna’s  Post  receives  positive  expected  utility  only  if  the  average  outcome  of 
attempting  to  destroy  enemy  posts  of  this  kind  has  been  success.  Once  again,  this 
aggregated  information  does  not  take  the  specifics  of  the  situation  into  account.  There 
may  be  factors  that  make  destroying  the  enemy  less  or  more  feasible  than  usual  in  this 
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particular  situation.  This  information,  if  its  exists  in  long-term  memory,  is  not  active 
in  the  current  network. 

INTERACTION  BETWEEN  REFLEXIVE  AND  REFLECTIVE  SYSTEMS 

Shruti  provides  a  mechanism  for  shifting  attention,  and  for  the  activation  of 
additional  information  in  long-term  memory-  by  means  of  such  shifting.  Figure  3  shows  the 
result  of  shifting  attention,  under  a  metacognitive  control  process,  from  the  observational  inputs 
to  one  of  the  four  assumptions  in  Figure  2.  The  nodes  labeled  Bl,  B2,  and  B3,  represent 
spreading  activation  during  the  second  attentional  cycle.  In  this  example,  the  decision  maker 
focuses  on  the  recognitional  response,  destroying  the  enemy  in  Sanna’s  Post.  This  shift  brings 
into  view  knowledge  that  was  previously  dormant.  The  newly  activated  knowledge  concerns  the 
feasibility  of  destroying  the  enemy  in  Sarna’s  Post.  Attention  shifting  disaggregates  the 
historical  information  stored  at  the  node  (destroying  enemy  in  Sanna’s  Post),  by  exploring  the 
contents  of  the  network  beyond  that  node.  In  doing  this,  it  allows  the  pattern  of  activation&to 
adapt  to  the  facts  of  the  present  situation  that  are  represented  in  the  newly  activated  part  of  the 
network.  Attention  shifting  thus  removes  some  of  the  reliance  in  decision  making  on  “habit,”  i.e., 
historically  aggregated  information. 

It  turns  out  that  the  facts  in  this  example  are  not  as  clear  cut  as  the  average.  On  the  one 
hand,  Sanna’s  Post  is  a  logistics  post,  which  is  typically  a  weakly  defended  target.  On  the  other 
hand,  armored  vehicles  have  been  spotted  there,  which  outgun  an  infantry  company.  This 
represents  a  conflict  of  evidence  about  the  feasibility  of  destroying  the  enemy  at  Sanna’s  Post. 
Here  we  have  a  typical  result  of  metacognitive  critiquing  and  correcting  (Cohen  et  al.,  2000a): 
the  solution  to  one  problem  (confirming  or  disconfirming  the  reliability  of  an  assumption)  leads 
to  another  problem  (conflicting  evidence). 

Determining  when  and  where  to  shift  attention  so  as  to  obtain  a  more  detailed  verification 
of  current  recognitional  conclusions  is  a  key  function  of  metacognitive  strategies.  In  Figure  4,  the 
commander  has  initiated  a  third  attentional  cycle,  by  shifting  attention  to  one  of  the  sources  of 
the  conflict:  the  presence  of  armor  at  Sanna’s  Post.  The  nodes  labeled  Cl,  C2,  and  C3  represent 
spreading  activation  in  the  third  attentional  cycle. 

Shruti  exhibits  priming  effects  that  preserve  and  integrate  the  results  of  reflexive 
reasoning  during  successive  shifts  of  attention.  The  new  information  activated  by  an  attention 
shift  must  be  combined  with  information  that  was  active  before  the  shift.  Once  the  decision 
maker  has  shifted  attention  to  the  armor  at  Sanna’s  Post  in  Figure  4,  most  of  the  initially  active 
part  of  the  network  (in  Figure  2)  slides  beyond  the  range  of  activation.  In  Figure  4,  destroying 
the  enemy  at  Sanna’s  Post  is  now  on  the  edge  of  the  active  network  again,  but  the  consequences 
of  this  action  (Figure  2)  are  now  inactive.  Its  expected  utility,  however,  is  not  simply  an  average 
over  many  occasions  of  destroying  such  posts.  It  reflects  details  of  the  present  situation  that  were 
very  recently  active  in  Figure  2  and  that  are  now  primed.  This  primed  information  shows  the 
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relevance  of  Sanna’s  Post  to  the  battalion  fight  to  the  east.  As  a  result,  the  expected  utility  of 
destroying  the  enemy  at  Sanna’s  Post  is  far  higher  than  a  historical  average  would  indicate. 
Without  priming,  reflexive  reasoning  would  be  a  succession  of  hermetically  sealed  windows, 
uninfluenced  by  context  except  through  crystallized  historical  averages.  With  priming,  the 
context  of  each  active  window  exerts  a  real-time  influence  through  primed  information,  even 
when  it  is  not  fully  attended. 

\  \ 


input 


Figure  3:  The  result  of  shifting  attention  to  the  action  of  destroying  the  enemy  at  Sanna’s  Post  (shaded 
node).  Nodes  labeled  Bl,  B2,  and  B3  reflect  the  order  of  activation  after  the  attention  shift. 
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figure  4:  Result  of  shifting  attention  again,  to  belief  that  we  are  outgunned  by  tanks  at  Sanna’s  Post 
(shaded  node).  Labels  Cl,  C2,  and  C3  represent  ordering  of  activation  of  nodes  after  this  attention  shift. 

Reflective  and  reflexive  attitudes  can  co-exist  although  they  compete  for  some  of  the 
same  resources.  In  order  to  detect  uncertainty  in  a  mental  model,  the  decision  maker  must  adopt 
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a  reflective  stance.  This  means  simply  that  he  or  she  is  treating  at  least  one  of  the  components  of 
the  currently  active  reflexive  network  (e.g.,  instantiated  relations)  as  an  object.  Shruti  constrains 
the  total  number  of  objects  that  can  be  discriminated.  As  a  result,  the  more  reflective  a  decision 
maker  becomes,  the  less  able  he  or  she  is  to  sustain  dynamic  discrimination  of  the  objects  in  the 
domain,  such  as  ships  and  countries.  Because  of  this  constraint  on  the  total  number  of  objects, 
during  routine  processing  reflection  will  tend  to  be  limited  to  the  predicate  in  focal  attention.  In 
novel  or  uncertain  situations,  reflection  can  expand  to  include  more  parts  of  the  active  network 
and  to  represent  evidence-conclusion  relationships  among  these  objects.  Thus,  the  reflective 
stance  (which  objectifies  statements)  competes  with,  but  can  also  co-exist  with,  the  reflexive 
stance  (which  objectifies  objects  in  the  domain).6 

The  metacognitive  system  recognizes  qualitatively  different  patterns  of  uncertainty 
in  the  active  network.  Decision  makers  respond  differently  when  the  probability  of  events  is 
known  (e.g.,  50%  chance  of  heads  and  50%  chance  of  tails  in  a  coin  toss)  than  when  the 
probabilities  are  totally  unknown  (e.g.,  which  of  two  unfamiliar  tennis  players  will  win  a  match; 
Ellsberg,  1988).  Still  another  situation  arises  in  which  strong  evidence  is  brought  forward  both 
for  a  hypothesis  and  against  it  (e.g.,  two  well-supported  theories  which  give  different  predictions 
for  a  particular  experimental  outcome).  It  seems  plausible  to  suppose  that  the  first  case  involves  a 

different  type  of  uncertainty  (risk)  than  the  second  case  (ambiguity  or  ignorance)  or  the  third 
(conflicting  evidence). 

In  real-world  decision  making,  people  do  not  reduce  all  types  of  belief  and  uncertainty  to 
a  single  measure.  Yet  traditional  probabilistic  models  have  been  dominated  by  single  measures 
(such  as  probability  as  a  measure  of  belief,  and  entropy  as  a  measure  of  uncertainty7).  Shruti 
provides  a  richer  vocabulary,  and  a  capability  for  more  naturally  representing  a  variety  of 
uncertainty-handling  strategies.  The  basis  for  this  flexibility  is  Shruti’s  independent  registration 
of  evidence  for  and  against  a  hypothesis  and,  analogously,  the  reasons  for  and  against  performing 
an  action.  This  enables  the  metacognitive  system  to  discriminate  four  qualitatively  distinct 
uncertainty  patterns  that  may  exist  at  a  single  node  at  a  given  time: 

■  Conclusion  or  decision-,  significant  activation  either  for  or  against  a  hypothesis  but 
not  both; 

■  Incompleteness  of  information:  little  or  no  activation  either  for  or  against  a  hypothesis 
(corresponding  to  ignorance  or  ambiguity) 

■  Lack  of  resolution:  moderate  amounts  of  activation  both  for  and  against  a  hypotheses, 
in  which  the  sum  of  +  and  -  activation  is  one  or  less  (e.g.,  the  coin  toss) 


Mother  factors  that  promotes  co-existence  is  that  the  object  discriminations  that  have  been  made  reflexively  are 
Entropy  is,  roughly,  the  degree  to  which  probabilities  of  each  event  approximate  1/n  for  n  possible  events. 
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■  Conflicting  information:  strong  activation  both  for  and  against  a  hypothesis,  in  which 
the  sum  of  +  and  -  activation  is  greater  than  one. 

In  addition,  by  adding  the  temporal  dimension,  we  get  a  fifth  pattern: 

■  Dependence  on  assumption :  activation  that  is  subject  to  change  as  more  information 
is  considered  (implicit  assumption),  or  as  different  choices  are  made  (explicit 
assumption). 

Attending  to  a  belief  reflectively  results  in  recognition  of  uncertainty  patterns  pertaining  to  that 
belief.  In  addition,  the  decision  maker  may  employ  more  proactive  strategies  for  uncovering 
uncertainty.  Such  strategies  involve  clamping  truth  values  in  order  to  consider  hypothetical 
situations  and  plans,  i.e.,  what  if  reasoning.  These  hypothetical  beliefs  and  actions  may  activate 
relevant  information  that  could  not  otherwise  be  considered.  This  information  in  turn  may  lead  to 
recognition  of  additional  gaps,  conflicts,  and/or  assumptions. 

The  three  major  types  of  uncertainty  (incompleteness,  conflict,  and  assumptions)  all 
appear  in  the  Sanna’s  Post  example. 

Incompleteness 

In  Figure  2,  movement  of  armored  vehicles  on  the  road  has  been  predicted.  Nevertheless, 
this  node  may  represent  a  gap,  since  its  strength  of  activation  may  be  quite  weak  until  the 
movement  has  actually  been  observed.  Other  gaps  are  represented  by  other  components  of  the 
intent  mental  model  that  have  not  participated  in  the  arguments  for  or  against  enemy  intent.  In 
particular,  although  enemy  capability  relative  to  friendly  forces  has  received  some  activation,  no 
specific  information  has  been  retrieved.  Such  information  could  have  important  consequences. 

For  example,  if  the  enemy  expected  us  to  be  stronger  in  this  sector,  they  might  choose  to 
reinforce  the  battalion  fight  by  some  other  route. 

A  more  proactive  method  for  uncovering  gaps  is  to  shift  attention  from  evidence  (the 
presence  of  trucks)  to  conclusions  (e.g.,  the  enemy  intends  to  use  Sanna’s  Post  as  a  logistics 
base),  and  clamp  the  conclusion  true  -  that  is,  ask,  what  if  the  conclusion  is  true ?  This°query 
results  in  the  reflexive  propagation  of  activation  that  is  equivalent  to  two  questions: 

■  How  can  the  conclusion  be  explained?  The  attention  shift  may  draw  attention  to 

additional  gaps  in  the  mental  model  representing  causes  of  the  intent.  For  example, 
the  commander  has  considered  the  element  of  opportunity  as  it  affects  the  enemy 
(e.g.,  the  road),  but  not  very  thoroughly.  He  has  not  thought  about  other  possible 
routes  the  enemy  might  use,  and  whether  they  offer  any  advantages,  e.g.,  are  they  less 
vulnerable  to  ambush?  J 

■  What  does  the  conclusion  predict?  The  attention  shift  may  help  generate  new 
predictions  regarding  expected  enemy  actions  that  can  be  used  to  test  the  assessment 
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Conflict 


In  Figure  3,  conflicting  arguments  are  activated  about  whether  or  not  Sanna’s  Post  is 
vulnerable  to  an  attack.  On  the  one  hand,  logistics  bases  are  likely  to  be  weakly  defended.  On  the 
other  hand,  enemy  armor  is  present  at  Sanna’s  Post. 

Strategies  for  filling  gaps  can  always  lead  to  conflict,  and  so  they  double  as  strategies  for 
finding  conflict.  More  specific  and  more  proactive  strategies  for  discovering  conflict  are  also 
available.  To  discover  potential  conflicting  evidence,  the  commander  can  clamp  the  current 
conclusion  as  false,  or  the  current  plan  as  failing  to  achieve  its  objectives.  For  example,  to  find 
out  if  there  is  any  evidence  for  an  alternative  explanation  of  the  trucks  at  Sanna’s  Post,  the 

commander  can  imagine  that  he  knows  that  the  enemy’s  intent  is  not  to  use  Sanna’s  Post  as  a 
logistics  base.  This  attention  shift  is  equivalent  to  asking  two  questions  of  the  reflexive  system: 

■  Flow  or  why  is  the  conclusion  could  be  false?  This  may  activate  knowledge  of 
alternative  possible  explanations  for  the  trucks.  For  example,  the  enemy  might  wish 
to  deceive  us  in  order  to  fix  the  company  and  prevent  us  from  joining  the  battalion 
fight  ourselves. 

■  What  does  the  falsity  of  the  conclusion  predict?  The  alternative  explanation  may  lead 
to  other  predictions  (e.g.,  regarding  overall  enemy  strength  and  recent  losses),  which 
may  be  verified  by  further  information  collection. 

Assumptions 

We  have  already  seen  that  a  particularly  important  kind  of  assumption  concerns  beliefs  at 
the  edge  of  the  currently  active  network:  Does  the  historically  aggregated  information  actually  fit 
the  current  situation,  or  will  conclusions  change  as  more  specific  information  about  the  present 
situation  is  considered?  Figure  2  includes  four  different  assumptions:  regarding  the  prior 
probabilities  of  enemy  purpose  and  opportunity,  the  expected  utility  of  the  enemy’s  moving 
reinforcements  along  the  road  through  Sanna’s  Post,  and  the  feasibility  of  destroying  the  enemy 
in  Sanna  s  Post.  In  each  of  these  cases,  it  is  possible  that  past  experience  was  non-representative, 
or,  conversely,  that  the  present  situation  is  unusual. 

A  simple  way  to  discover  assumptions  begins  by  recognizing  the  heightened  possibility 
of  dependence  on  assumptions  for  beliefs  at  the  edge  of  the  current  reasoning  process.  These  are 
beliefs  which  have  simply  been  accepted  as  true,  and  are  not  embedded  in  a  web  of  reasoning  or 
observation.  The  decision  maker  can  shift  attention  to  such  nodes,  and  determine  whether  the 
degree  of  belief  in  the  event  represented  by  the  node  changes  as  a  result  of  newly  activated 
information.  Such  newly  activated  information  may  represent  plausible  causes  of  the  event,  or 
testable  predictions  implied  by  the  event. 

A  context  in  which  assumptions  are  especially  important  is  in  the  effort  to  resolve 
conflict.  Since  a  proposition  and  its  contradiction  cannot  both  be  true,  conflict  itself  is  a  strong 
cue  that  assumptions  somewhere  in  the  system  of  belief  are  false  (Cohen,  1986).  The  stronger  the 
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conflict,  the  more  likely  it  is  that  one  or  more  of  the  beliefs  responsible  for  the  conflict  must 
depend  on  assumptions  that  are  false  in  the  present  situation.  Such  assumptions  may  be 
uncovered  by  shifting  attention  to  beliefs  which  contribute  to  the  conflict. 

A  more  proactive  strategy  for  ferreting  out  hidden  assumptions  is  similar  to  the  strategy 
for  finding  conflict.  This  involves  clamping  a  conclusion  as  false  and  the  evidence  for  the 
conclusion  as  true.  In  response,  the  system  reflexively  searches  for  alternative  explanations  of 
the  evidence.  If  these  alternatives  turn  out  to  represent  gaps,  i.e.,  there  is  little  evidence  for  or 
against  them,  then  the  original  conclusion  depended  on  the  implicit  assumption  that  these 
alternative  explanations  were  false. 

The  metacognitive  system  increases  its  effectiveness  by  learning  strategies  that  are 
specifically  tailored  to  each  type  of  uncertainty. 

The  metacognitive  system  learns  to  combine  a  set  of  simple  operations:  inhibiting 
recognitional  responding,  activation  of  new  information  internally  by  shifting  focal 
attention,  and  clamping  truth  values.  These  operations  are  simple  and  both  psychologically 
and  biologically  plausible.  The  metacognitive  system  learns  to  combine  these  operations  in 
response  to  different  patterns  of  uncertainty  by  reinforcement  and  associative  learning  processes 
Through  such  learning,  the  metacognitive  system  acquires  a  rich  repertoire  of  uncertainty 
handling  strategies  based  both  on  knowledge  acquired  about  the  specific  domain,  and  on  more 
generalizable  principles. 

Strategies  utilized  by  the  metacognitive  system  contain  both  domain-specific  and 
general-purpose  elements,  to  varying  degrees  as  a  function  of  experience.  After  extensive 
experience  in  a  domain,  decision  makers  learn  which  concepts  are  likely  to  be  the  major 
determinants  of  different  kinds  of  uncertainty  for  other  concepts.  In  other  words,  they  Ieam  to 
identify  likely  culprits,  i.e.,  nodes  in  the  active  network  that  are  likely  to  be  responsible  for  the 
gap,  conflict,  or  unreliable  assumption  that  has  been  identified.  This  might  include,  for  example, 
knowledge  of  what  the  typical  gaps  in  understanding  are  likely  to  be  in  estimating  enemy  intent, 
what  the  likely  causes  of  conflicting  conclusions  about  enemy  intent  are,  and  where  the  hidden 
assumptions  lie.  These  learned  associations  guide  reflective  processing.  They  bring  with  them 
both  an  increase  in  efficiency  and  the  risk  that  novel  sources  of  uncertainty  will  be  overlooked 

In  filling  gaps  and  in  resolving  conflict,  the  metacognitive  system  utilizes  measures  of 
culpability  that  reflect  the  sensitivity  of  belief  in  one  node  (e.g.,  the  node  where  there  is  a  gap  or 
conflict)  to  changes  of  belief  in  other  nodes.  These  are  closely  related  to  the  parameters  that  are 
used  to  tune  weights  in  the  reflexive  system  to  environmental  correlations.  The  result  is  mastery 
of  domain-specific  strategies  for  metacognitive  critiquing  and  attention-shifting. 

More  general  strategies  for  reflective  reasoning  may  also  be  learned,  by  abstracting  from 
experience  or  by  explicit  instruction.  These  general  strategies  identify  likely  causes  and  cures  for 
different  types  of  uncertainty  by  using  argument  relationships.  For  example,  decision  makers 
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may  learn  than  when  a  conflict  in  beliefs  is  discovered,  they  should  shift  attention  to  evidence  for 
the  conflicting  conclusions . 

Our  discussion  of  Figure  2  illustrates  a  more  general  strategy.  The  reflective  system 
recognized  that  the  decision  to  destroy  the  enemy  at  Sanna’s  Post  might  well  depend  on 
assumptions,  since  the  chain  of  reasoning  that  led  to  that  response  had  not  been  thought  through 
very  deeply.  As  a  result  of  this  uncertainty  and  the  high  stakes  of  the  decision,  the  reflective 
system  inhibited  the  initial  recognitional  response.  Instead  of  acting  reflexively  on  that  response, 
the  reflective  system  identified  it  as  the  conclusion  and  shifted  attention  to  it  in  order  to  evaluate 
the  assumptions  upon  which  it  depended,  The  aim  was  to  explore  the  chain  of  reasoning  more 
deeply  and  expose  information  beyond  the  edge  of  the  currently  active  network.  Exposure  of  this 
information  resulted  in  the  discovery  in  Figure  3  of  a  conflict  between  evidence  that  Sanna’s 
Post  will  be  vulnerable  to  attack  and  evidence  that  it  will  not. 

The  grounds  of  both  the  optimistic  and  the  pessimistic  argument  are  at  the  edge  of  the 
current  network,  and  thus  likely  to  be  dependent  on  implicit  assumptions  about  the 
representativeness  of  historically  aggregated  information.  To  resolve  the  conflict,  in  Figure  4  the 
commander  shifts  attention  to  the  conclusion  of  one  of  the  two  competing  arguments  (i.e.,  that 
we  are  outgunned  by  tanks  at  Sanna’s  Post).  The  purpose  of  this  attention  shift  is  to  activate 
assumptions  that  may  turn  out  not  to  fit  this  situation,  that  is,  to  expose  incorrect  generalizations 
about  the  strength  of  this  defense.  Activation  of  new  information,  if  it  results  in  revision  of 
assumptions,  may  eliminate  the  conflict.  If  this  fails,  the  decision  maker  might  shift  attention  to 
the  conclusion  of  the  other  competing  argument. 

Metacognition  helps  guide  the  dynamic  retrieval  and  collection  of  new  information,  and 
facilitates  learning.  It  thus  helps  create,  maintain,  and  improve  the  belief  network. 

Value  tradeoffs  contribute  to  the  control  of  reflective  processing.  Recognition-based 
and  reflective  responding  fall  along  a  spectrum  that  varies  the  number  of  attentional  cycles 
enlisted  to  arrive  at  a  conclusion  or  response.  Decisions  about  whether  to  reflect  more  (i.e., 
engage  in  additional  attentional  cycles)  or  act  at  once  on  the  current  best  response  are  determined 
by  the  current  uncertainty  of  recognitional  conclusions,  the  costs  of  delay,  and  the  potential  costs 
of  errors. 

CONCEPTS  FOR  UNCERTAINTY  VISUALIZATION 

A  graphical  user  interface  for  the  Reflexive-Reflective  System  has  been  implemented 
with  the  following  features:  The  GUI  provides  a  graphical  display  of  the  inferential  relationships 
in  the  network  of  beliefs  and  actions  that  represents  the  situation.  The  GUI  dynamically  displays 
the  current  uncertainty  status  of  each  active  node  in  working  memory,  in  terms  of  qualitatively 
different  types  of  uncertainty.  The  GUI  supports  domain-specific  critical  thinking  strategies  in 
real  time,  by  providing  information  about  the  degree  of  historical  culpability  of  each  node  in  the 
network  for  uncertainty  at  other  nodes.  The  GUI  supports  general-purpose  critical  thinking 
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strategies  in  real  time,  by  providing  information  about  the  generic  roles  that  nodes  are  currently 
playing  within  arguments  that  bear  on  crucial  uncertainties  in  the  current  situation. 

SUMMARY 

In  sum,  there  are  inherent,  and  dynamic,  limits  on  the  scope  of  LTM  information  that  can 
be  brought  to  bear  in  interpreting  any  evidence.  The  key  interaction  between  the  reflexive  and 
reflective  systems  is  the  adaptive  direction  of  focused  attention  within  the  reflexive  memoty  by 
means  of  learned  metacognitive  behaviors.  Recency  effects  are  used  to  assemble  such 
intermediate  results  into  composite  assessments.  The  model  suggests  that  the  development  of 
executive  attention  functions  (metacognitive  strategies)  may  be  necessary  for,  and  integral  to,  the 
development  of  working  memoiy,  or  dynamic  access  to  LTM.  A  simulation  tool  with  these 

properties  may  be  able  to  provide  the  kind  of  dynamic  and  flexible  feedback  and  guidance  that  is 
necessary  in  training  critical  thinking  skill. 
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Results  of  Group  Discussions  in  Critical  thinking 

This  section  summarizes  the  results  of  major  themes  addressed  in  the  discussion  groups 

and  ends  with  recommendations  for  future  research.  The  material  is  divided  into  three 
subsections: 

■  Critical  thinking  in  battle  command 

■  Training  critical  thinking  in  the  army 

■  Critical  research  and  development  issues  in  training  critical  thinking. 


CRITICAL  THINKING  IN  BATTLE  COMMAND 

Use  of  Critical  Thinking  in  Battle  Command 

Battle  Command  situations  are  very  complex  with  high  stakes — e.g.,  people’s  lives. 
They  are  deliberately  deceptive  in  nature  with  the  hostilities  of  the  enemy  at  the  forefront  of  our 
deliberations.  We  have  to  work  in  a  hierarchy.  We  have  to  deal  with  international  public 
perceptions.  It’s  done  in  a  political  atmosphere  with  different  agendas  of  different  groups. 
There’s  a  spectrum  of  operations  from  peace  keeping  to  combat.  There  will  be  operations  we 

cannot  now  imagine,  let  alone  prepare  for.  Rapid  advances  in  technology  make  decisions  more 
complex. 

We  can’t  solve  these  complex  problems  without  critical  thinking.  We  especially  need 
critical  thinking  to  deal  with  novel  situations.  We  cannot  train  for  novel  situations  because  we 
do  not  know  what  those  situations  will  be.  Even  with  simulations  we  can’t  simulate  all  possible 

situations.  So  we  must  train  to  deal  with  novel  situations  using  critical  thinking  skills  (critical 
thinking  skills). 

Battle  Command  tasks  that  particularly  relate  to  critical  thinking  are: 

■  Visualizing  operations 

■  Describing  outcomes  and  projecting  them 

■  Directing  the  operation  with  subordinates, 

■  Knowing  when  to  stop  and  to  start  planning 

■  Assessing  evidence 

■  Questioning  assumptions 

■  Envisioning  in  time 

■  Keeping  your  eye  on  the  goal 

■  Knowing  when  to  start  and  stop  critical  thinking 
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Relationship  between  The  Military  Decision  Making  Process  (MDMP)  and  Critical 
Thinking 

Participants  agreed  that  the  MDMP  and  critical  thinking  are  different  but  fit  together. 
Critical  thinking  is  one  of  the  means  by  which  the  MDMP  can  be  executed  and  using  critical 
thinking  can  improve  the  quality  of  the  products  of  the  MDMP.  Following  are  some  of  the 
comments  on  this  topic. 

There  is  no  conflict  between  MDMP  and  critical  thinking  because  they  are  different  and 
they  compliment  each  other.  The  MDMP  could  be  seen  as  a  meta-language  to  communicate  the 
results  of  the  thinking  process.  The  MDMP  is  a  process  while  critical  thinking  is  a  modifier  to 
the  process.  This  means  you  need  to  know  the  process  before  you  can  use  critical  thinking. 

Critical  thinking  is  used  in  executing  the  process  and  should  be  integrated  into  the 
MDMP.  We  need  to  figure  out  how  to  apply  critical  thinking  to  MDMP.  The  Army  uses 
visualization,  maps  and  mental  simulation.  When  critical  thinking  is  taught,  it  must  be  mapped 
into  these  procedures  and  in  fact  mapped  into  the  whole  MDMP. 

Our  planning  assumptions  have  values  that  appear  to  be  precise,  but  they  are  not.  This  is 
another  place  where  we  need  to  apply  critical  thinking.  We  can  also  use  critical  thinking  to  adjust 
the  MDMP  to  time  available,  i.e.  to  modify  the  MDMP. 

Incidentally,  the  MDMP  reflects  another  training  problem.  We  teach  officers  that  the  first 
step  is  an  analysis  of  your  mission  and  that  you  break  the  mission  into  its  components,  specified 
and  implied  tasks.  Then  you  assign  them  into  the  stovepipe  planning  cells.  They  are  kept  safely 
m  the  pipeline  and  the  only  person  responsible  for  any  synthesis  in  the  process  is  the 
commander.  So  one  of  the  problems  is  that  there  is  a  group  responsible  for  breaking  things  into 
small  parts  and  only  one  person  responsible  for  synthesis. 

The  Relationship  Between  Critical  Thinking  and  Intuition 

This  topic  generated  disagreement,  with  some  participants  arguing  that  intuition  is  the 
result  of  critical  thinking  and  therefore  is  analytical,  and  others  arguing  that  it  may  or  may  not  be 
the  result  of  critical  thinking,  but  is  not  analytical.  Following  are  selected  arguments  made  by 
discussants: 

■  I  would  argue  that  intuition  is  a  result  of  critical  thinking.  You  go  through  multitudes 
of  critical  thought  processes  about  an  issue  and  these  are  then  stored  in  memory.  Then 
you  use  some  kind  of  analogical  process  to  retrieve  that  information.  It  comes  to  our 
mind  as  something  we  call  intuition  and  in  fact  it  is  analytic,  very  rational,  and  the 
product  of  processes  that  have  taken  place  at  a  previous  time. 

■  What  often  happens  is  the  first  thing  that  comes  to  conscious  awareness  is  the 
solution.  Intuition  is  an  extremely  appropriate  tool  of  reasoning  because  critical 
thinking  is  linear,  conscious,  and  requires  a  lot  of  energy.  The  unconscious  rapid 
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processing  can  happen  in  parallel,  takes  into  account  a  whole  lot  of  variables  based  on 
experience,  and  comes  up  with  the  solution. 


Well  I  disagree.  It’s  non-analytic.  Recognition  prime  decision-making  at  its 
simplest  level  is  really  non-analytic.  It’s  rational,  but  that  doesn’t  mean  it’s  analytic. 

I  agree  with  that.  Intuition  may  be  the  product  of  critical  thinking.  In  fact,  we  talked 
about  intuition  as  an  important  element  in  planning.  That  is,  applying  critical 
thinking  to  planning,  such  that  when  you  get  to  the  actual  execution,  and  you’re  in  the 
throes  of  events,  you  will  have  worked  through  those  ideas  before.  You  will  have 
foreseen  the  issues  that  might  come  out,  because  you’ve  spent  so  much  time  planning, 
bo  that  when  a  situation  occurs,  you  recognize  the  solution  and  all  the  elements.  And 
then  when  the  execution  is  there  for  you,  you  know  what  it  involves.  The  process  by 
which  you  recognize  and  execute  is  not  an  analytic  process.  But  it  may  depend  on  all 
the  critical  thinking  you  did  previously. . 


■  will  say  that  the  truth  is  on  both  sides  here,  it’s  non-analytic  in  the  sense  of  intuition 
not  being  a  serial  conscious  process,  but  there  are  incredible  inferential  powers  that 
are  unconscious.  If  you  see  Tom  give  Mary  a  book  and  somebody  asks  you  who  has 
the  book:  it’s  Mary.  All  this  information  is  compiled  in  parallel  and  converges  on  a 
rapid  intuitive  answer.  It  can  probably  combine  tens  of  thousands  of  bits  of 
information  in  parallel.  But  it  is  highly  inferential  in  the  sense  that  you  can  only 

model  it  using  logic  or  something  like  that.  But  it’s  certainly  not  a  conscious  thought 
process. 


■  The  phenomenon  of  intuition  on  a  battlefield,  they  describe,  has  been  discussed  in 
military  theory.  They  first  called  it  ’‘In  my  mind's  eye.”  Germans  call  it  “Finger  tip 
feel.  Americans  call  it:  “Good  eye  for  battlefield.”  Coming  back  to  Burt’s  opening 
comments:  Pattern  recognition  may  be  a  part  of  this.  If  you  have  a  visual  feel  for 
what  should  be  happening  and  if  it  isn’t  happening  you  can  cause  it  to  happen  by 
influencing  something.  If  on  the  other  hand,  if  something’s  happening  that  you  know 
shouldn  t  happen,  you  could  do  something  about  it.  This  is  intuition.  I  would  argue 
that  this  intuition  is  based  on  experiences.  You  can  teach  it  -  you  can  teach  peopfe 
how  to  have  a  better  feel  for  the  battlefield  by  putting  them  through  dealing  with 
experiences.  I  would  also  argue,  and  this  is  a  very  anti-American  view,  maybe  there’s 
some  talent  involved  and  some  people  are  never  going  to  have  an  eye  for  the 
battlefield. 


■  Intuition  could  be  a  result  of  critical  thinking.  Having  gone  through  multitudes  of 
critical  thinking  processes  that  get  stored  away,  analogous  thinking  brings 
conclusions  forward  automatically.  Recognition  Primed  Decision  Making  (RPDM) 
is  intuition  according  to  Gary  Klein.  It  is  rational  but  not  analytical.  Information  is 
encoded  for  intuitive  rapid  processing.  RPDM  is  a  parallel  process  to  critical  thinking. 
It  s  not  critical  thinking,  but  we  run  these  two  on  a  continuing  basis  when  we’re 

trying  to  make  decisions.  RPDM  is  an  important  process.  Military  commanders  use  it 
a  lot,  especially  experienced  ones. 
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We  need  to  take  leaders  and  decision  makers,  put  them  in  increasingly  complex  and 
varied  scenarios,  have  them  make  decisions,  and  give  them  feedback  on  their 
decision.  And  that’s  all  rational  and  analytic.  But  as  we  develop  true  decision 
makers  there  s  some  automaticity  there.  If  you  create  situations  either  through 
simulation  or  analogy,  and  you  continually  put  people  through  them  and  question 
t  em  and  make  them  do  critical  thinking,  I  think  you  build  intuition.  So  the  pattern 
recognition,  the  intuition,  starts  to  grow  and  is  based  on  critical  thinking. 

BARRIERS  TO  CRITICAL  THINKING 

Following  are  factors  the  group  thought  were  especially  detrimental  to  conducting  critical 
thinking. 

Mental  Effort 

Fatigue  and  excessive  mental  workload  can  stop  critical  thinking  from  occurring,  critical 
thinking  is  hard  work  and  can  be  an  unpleasant  state  to  be  in. 

Critical  thinking  is  an  arduous  procedure  on  two  levels.  First,  there  is  the  mental  work 
that  is  done  at  the  individual  and  team  levels.  Second,  the  prospect  of  having  to  change  decisions 
and  judgments  as  a  result  of  critical  thinking  can  discourage  critical  thinking.  Knowing  how  to 

manage  one’s  cognitive  resources  to  meet  the  demands  of  the  task  is  an  important  part  of  being 
able  to  think  critically. 

Social  awkwardness  can  discourage  critical  thinking 

If  one  is  concerned  about  the  social  acceptance  of  people  who’s  work  you  are  critiquing, 
this  may  discourage  critical  thinking.  Social  skills  are  important  when  using  critical  thinking 
skills  in  social  and  team  settings.  We  need  to  teach  social  skills  when  using  critical  thinking  so 
we  don’t  alienate  people.  But  if  we  make  routine  the  use  of  critical  thinking,  then  critical 

thinking  becomes  part  of  the  process  and  is  expected.  Then  people  will  then  not  feel  attacked  if 
their  work  is  critiqued. 

Army  Cultural  Climate 

The  command  climate  can  either  nurture  or  stifle  critical  thinking.  We  need  to  create  a 
culture  in  which  officers  can  think  critically.  But  this  is  not  only  a  problem  for  the  Army;  it’s  a 
problem  for  most  large  organizations,  critical  thinking  is  impossible  to  achieve  unless  the  culture 
supports  it.  We  like  to  think  rational  argument  will  carry  the  day.  But  the  political  and  cultural 
climates  are  too  important.  The  organizational  culture  can  be  an  impediment  to  creating  an 
environment  in  which  it’s  possible  to  think  critically.  The  hierarchical  authority  structure 

discourages  a  younger  officer  with  coming  up  with  a  novel  idea  that  may  be  at  odds  with  his 
commanding  officer’s  ideas. 
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Lack  of  Time 


Critical  thinking  requires  a  certain  amount  of  time  to  execute.  It's  not  instantaneous.  If 
t  ere  is  not  enough  time,  critical  thinking  may  default  to  intuition. 

TRAINING  CRITICAL  THINKING  SKILLS  IN  THE  ARMY 

How  do  we  currently  develop/  train  critical  thinking  in  Army  Officers? 

Current  critical  thinking  training  in  officers  uses  the  three  pillars  of  the  professional  development 
model  institutional  training,  self-development  and  operational  assignments. 

Institutional  Training 

Here  we  have  small  groups,  seminars,  trained  instructors,  retired  officer  training 
facilitators,  and  simulations.  The  experiential  learning  model  is  used.  One  problem  here  is  that 

junior  people  are  given  course  design  responsibility  before  they  are  expert  in  applying  critical 
thinking  skills  to  planning. 

Operational  Assignments 

It  is  easier  to  teach  critical  thinking  in  the  field  because  we  have  more  freedom  After 
Action  Reviews  (AARs)  including  Observer  Controllers  is  a  successful  model.  The  emphasis 
here  is  on  finding  out  how  to  get  better  vs.  finding  the  right  answer.  Increasing  experience 
opportunities  can  be  used  to  shorten  the  learning  cycle. 

Self  Development 

While  professional  development  is  a  great  mechanism,  there  isn’t  much  in  there  that  talks 
about  critical  thinking.  If  we  want  to  get  people  to  think  more  about  critical  thinking  we  need  to 
make  sure  it’s  in  the  reading  lists  that  we  give  to  people.  On  the  other  hand,  you  don’t  learn  to 
apply  critical  thinking  by  studying  critical  thinking  in  theory  and  principle.  You  learn  by  doin« 
it.  If  we  really  want  to  make  critical  thinking  a  natural  way  of  thinking  we  need  to  show  them" 

how  they  can  apply  critical  thinking  in  all  kinds  of  contexts  -  at  home,  in  their  physical  fitness 
as  well  as  in  their  professional  lives. 

Where  does  distance  learning  fit?  In  terms  of  self-development,  we  are  moving  into  this 
e  ucation  mode  using  computers,  email  and  web  sites.  But  how  does  it  all  fit  together?  Do  we 
give  people  enough  time  to  take  advantage  of  these?  We  need  to  make  sure  people  have  the 
resources  they  need  to  do  self-development.  Perhaps  we  can  use  reading  lists  and  encourage 
books  about  critical  thinking.  And  how  do  we  reward  people  for  doing  that.  Right  now  the 
reward  system  does  not  support  those  who  do  self-development. 
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WHAT  WORKS  WELL  IN  HOW  WE  CURRENTLY  TRAIN  CRITICAL  THINKING  IN 
THE  ARMY? 

After  Action  Reviews 

This  can  be  a  very  effective  way  to  teach  critical  thinking.  The  Observer  controllers  give 
feedback  based  on  their  experience  and  formal  training  in  observation,  questioning  and 
interaction  skills.  The  Observer  controllers  should  think  about  how  they’re  going  to  get  critical 
thinking  feedback  to  the  soldiers,  and  what  looks  right. 

Performance  Based  Training 

Anything  that’s  performance  based  and  uses  coaching  can  be  effectively  used  for 
teaching  critical  thinking.  In  the  School  for  Command  Preparation  students  actually  go  through  a 
series  of  activities  and  critical  thinking  could  be  used  in  these  exercises  and  modeled  by  the 
instructors.  The  Common  Mission  Rehearsal  Exercises  at  the  CMTC  are  also  very  effective 
ways  to  train  critical  thinking  skills. 

Think  Like  a  Commander 

The  Think  Like  a  Commander  (TLAC)  vignette  program,  is  a  new  training  program  to 
train  adaptive  leaders  from  the  brigade  commander  down.  The  Initial  Brigade  Combat  Team  at 
Ft.  Lewis  has  a  program  similar  to  TLAC.  An  idea  in  TLAC  is  to  get  subordinates  to  think  like 
their  commanders,  to  think  at  a  higher  level.  TLAC  takes  a  vignette  to  the  next  level  and  creates 
a  simulation  where  it  presents  a  situation  to  a  commander  in  the  context  of  a  problem  that  they’re 
already  been  given.  But  now,  the  situation  is  something  different  than  they’ve  had  to  consider. 

He  is  then  asked  to  think  like  a  commander,  not  to  make  decisions  but  to  think  like  a 
commander.  He  works  through  his  interpretation  of  the  problem,  his  reasoning  and  the  types  of 
actions  he  might  take.  Then  we  relate  back  how  they  thought  about  it  to  how  great  commanders 
look  at  a  problem.  They  start  to  see  similarities.  After  4  or  5  vignettes  they  start  to  develop  a 
pattern  of  though  and  to  consider  areas  they  may  not  have  considered  in  earlier  vignettes.  They 
think  a  little  richer  and  deeper  about  the  process.  TLAC  focuses  on  critical  thinking.  This  could 
be  used  as  a  paradigm  for  training  critical  thinking. 

MAKING  THE  FISCHER  CRITICAL  THINKING  MODEL  MORE  RELEVANT  TO 
BATTLE  COMMAND 

There  was  a  great  deal  of  disagreement  about  whether  to  show  students  the  Fischer  model 
and  train  them  using  the  model.  Proponents  thought  showing  students  the  model  would  give 
them  the  big  picture  of  the  factors  affecting  critical  thinking  while  opponents  thought  use  of  the 

model  would  focus  students  on  the  theories  underlying  critical  thinking  rather  than  executing  the 
skills  themselves. 

Other  representative  comments  included: 
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■  The  model  is  too  static  to  represent  the  dynamic  thinking  that  goes  on  in 
visualization.  It  needs  to  be  enhanced  by  bringing  time  and  dynamicity  into  those 
skills  It  needs  a  goal  orientation  to  better  represent  the  military  dynamics  present  in 
the  planning  process. 

■  The  model  should  address  how  to  select  the  appropriate  cognitive  skill  relative  to  the 
requirements  of  the  problem. 

■  Intuition  is  not  included  in  the  model.  Officers  have  an  innate  sense  of  what  is  right 
and  wrong  based  on  experience.  When  a  battalion  commander  has  20-25  years  of 
experience,  you  ought  to  be  able  to  depend  on  him  to  make  intuitive  decisions  that  are 
correct.  Other  discussion  participants  disagreed  with  this  comment.  They  felt  that  if 

you’re  falling  into  intuitive  decisions  you  are  not  considering  other  alternatives.  You 
are  not  thinking  critically. 

■  The  use  of  maps  is  a  necessary  part  of  the  MDMP.  Visual  representations  are  tools  to 
aid  critical  thinking  dialog  and  tools  to  communicate.  How  can  we  support  critical 
thinking  mental  simulations  using  dynamic  maps?  How  can  we  blend  visualization 
and  critical  thinkings  in  the  model?  Visualization  and  mental  simulation  are 
equivalent  in  Army  planning. 


RECOMMENDATIONS  FOR  IMPROVING  CRITICAL  THINKING  TRAINING  IN 
THE  ARMY 


Capitalize  on  what  works  well  now. 


Routinely  incorporate  feedback  on  critical  thinking  in  the  formal  AARs.  The  AARs 
should  also  include  feedback  on  team  dynamics  in  critical  thinking  and  interpersonal  skills  using 
critical  thinking  in  teams.  Use  performance-based  training  with  coaching.  The  TLAC  program 
might  be  used  as  a  model  for  how  to  train  critical  thinking  skills. 

Design  digital  capabilities  to  aid  and  facilitate  critical  thinking. 


Training  critical  thinking  should  be  in  the  context  of  the  Army  digital  technology. 

Now  that  we  use  digital  tools  in  decision  making,  we  need  to  also  re-look  our  decision¬ 
making  processes.  With  technology  we  start  assigning  objective  values  to  things  that  should  be 
evaluated  subjectively.  When  these  objective  values  are  put  in  a  computer,  and  it  comes  up  with 
an  answer,  we  think  it  must  be  right.  The  values  we  deal  with  have  apparent  objective  values  and 
an  apparent  precision  they  may  not  have.  So  we  need  to  be  critical  and  evaluate  whether  the 
technological  process  is  appropriate  for  the  problem.  The  Army  needs  to  be  critical  about  the 
products  of  technology  and  the  information  products  of  technology.  Too  often  we  are  too 
accepting  of  the  products  of  technology.  This  is  a  place  for  critical  thinking  -  in  evaluating  the 
products  of  technology.  We  need  to  be  able  to  think  critically  about  technology. 


126 


Make  it  easy  for  the  instructors  to  train  critical  thinking. 

In  the  Leader  Instructor  Division  (LID)  of  CGSC,  they  came  up  with  ten  great  generic 
questions  which  could  be  embedded  in  the  lesson  plans,  e.g.  questioning  assumptions.  The 
Instructors  can  then  modify  these  base  questions.  They  can  be  used  in  the  AARs.  Develop 
similar  generic  questions  Instructors  can  use  for  training  critical  thinking. 

Train  critical  thinking  using  time  pressure. 

If  you  train  critical  thinking  with  no  time  pressure,  then  attempt  critical  thinking  under 
pressure,  it  may  break  down.  People  skip  over  steps  under  time  pressure  and  there  may  be  a 
default  to  intuition.  Undertime  pressure  there  is  less  time  to  think  through  something 
systematically.  Under  stress  you  may  have  other  cognitive  tasks  that  drain  the  cognitive 
resources  so  one  can’t  do  thorough  cognitive  processes.  Problem  solving  may  deteriorate  under 
high  stress.  But  if  critical  thinking  is  practiced  under  stress  and  time  stress,  officers  are  better 
prepared  to  do  critical  thinking  under  these  conditions. 

Give  guidance  on  when  to  use  and  when  not  to  use  critical  thinking. 

Teach  rapid  recognition/intuition  vs.  critical  thinking.  We  train  to  critically  think  when 
we  have  the  time  to  think.  But  knowing  when  not  to  use  critical  thinking  and  to  use  intuition 
instead  may  be  just  as  important  as  when  to  use  critical  thinking. 

Don’t  offer  answers. 

In  training  critical  thinking,  continue  to  focus  on  the  question  because  as  soon  as  you  allow 
or  accept  an  answer,  you  immediately  stop  all  thinking  about  that  particular  topic,  and  you’re 
ready  to  move  on  to  the  next.  Just  keep  asking  the  questions  and  go  deeper  and  deeper. 

Explicitly  train  the  process  of  critical  thinking. 

The  development  of  critical  thinking  skills  can  be  accelerated  by  making  the  process 
explicit.  Take  advantage  of  the  science  of  learning,  critical  thinking  skills  training  should  be 
planned  and  deliberately  inserted  in  courses,  and  not  left  to  chance.  However,  it  should  also  be 
incorporated  in  all  courses  by  having  instructors  model  how  critical  thinking  applied  to  the 
MDMP  and  reinforce  and  require  the  use  of  critical  thinking  in  their  students. 

Train  a  variety  of  critical  thinking  skills. 

Train  the  ability  to  select  an  appropriate  skill  for  the  situation.  If  people  have  a  toolbox  of 
critical  thinking  skills,  they  will  be  better  able  to  meet  the  unfamiliar  challenges  they  will  face  in 
the  future. 

Train  interpersonal  skills  for  the  application  of  critical  thinking. 

One  of  the  barriers  to  the  use  of  critical  thinking  is  social  awkwardness.  Even  though  one 
has  critical  thinking  skills,  if  one  is  not  able  to  use  them  in  group  settings  without  alienating 
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group  members,  those  skills  won’t  be  used, 
members. 


Use  of  them  will  be  punished  by  the  other  group 


Train  attitudes. 

Attitudes  about  critical  thinking  are  important  in  determining  whether  people  will  use 
critical  thinking  and  these  attitudes  can  be  trained.  Persistence,  open  mindedness  and  a 
willingness  to  expend  effort  are  necessary  in  using  critical  thinking. 

Train  for  transfer  of  training. 

Even  using  simulations,  we  can  t  simulate  all  possible  situations.  So  training  to  deal  with 
novel  situations  using  critical  thinking  skills  should  be  planned  for  and  part  of  the  course  design 

Methods  for  facilitating  transfer  of  training  to  other  situations  are  available  and  can  be  used 
when  designing  critical  thinking  training. 

Instructors  should  model  use  of  critical  thinking  skills. 

Training  should  also  show  students  what  critical  thinking  looks  like,  either  by  showing 
them  someone  who  is  critically  thinking  or  by  showing  them  a  way  to  walk  through  the  problem 
using  a  set  of  principles.  This  means  that  the  Instructors  must  be  good  critical  thinkers 
themselves  and  receive  training  and  feedback  themselves  in  critical  thinking. 

Include  critical  thinking  training  in  every  course  that’s  taught  in  the  Army. 

This  could  be  done  without  additional  resources.  This  is  currently  promoted/done  in 
universities.  You  can  take  any  course  (history,  math)  you’re  teaching  and  include  elements  that 
promote  critical  thinking.  Here’s  how  you  do  it.  This  is  just  what  Dr.  Halpem  talked  about 
yesterday.  You  don’t  ask  them  to  memorize  and  regurgitate  information.  You  ask  them  to  think 
about  it.  And  not  only  that,  but  the  Instructor  is  an  on-the-spot  assessment  and  evaluation 
instrument  of  that  critical  thinking.  Instructors  should  not  reinforce  students  when  students  just 
regurgitating  what  they  have  been  told  they  need  to  know,  but  instead  reinforce  working  through 
problems.  Instructors  can  ask  open-ended  questions  instead  of  multiple-choice  questions. 

On  the  other  hand,  another  discussant  thought  that  it’s  pretty  easy  to  include  the 
principles  and  standards  of  critical  reasoning  and  creative  thinking  in  task  standards  and  when 
developing  learning  standards.  It  takes  5  minutes.  But  to  get  someone  to  facilitate  his  classroom 
in  accordance  with  those  principles  takes  an  awful  lot  of  resources.  It  takes  training,  time, 
dedication,  someone  to  evaluate  what  the  Instructor  is  doing.  If  you’ve  got  200  instructors,  it’s 
very  difficult  to  infuse  them  with  the  desire  to  embed  critical  thinking  principles  throughout  their 
instruction.  It  sounds  pretty  easy,  but  it’s  very  difficult  to  accomplish.  Not  impossible,  and  well 
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Foster  an  Army  culture  that  values  critical  thinking. 

If  the  Army  fails  to  encourage  and  reward  creative  and  critical  thinking  within  its  officer 
and  NCO  corps,  further  research  and  training  efforts  in  the  area  of  critical  thinking  will  be 
wasted  efforts.  An  organizational  culture  with  norms  that  punish  critical  thinking  or  the 
behaviors  needed  to  develop  critical  thinking  skills  can  have  a  chilling  effect  on  the  development 
of  critical  thinking  within  the  organization — no  matter  how  much  lip  service  is  paid  to  the 
concept  of  critical  thinking  by  organizational  leadership.  For  example,  courses  on  critical 
thinking  can  be  incorporated  into  the  Army’s  institutional  training  curricula  and  can  be 
incorporated  into  training  doctrine.  However,  as  long  as  there  exist  leadership  positions  within 
the  Army  that  do  not  allow  their  subordinates  the  discretion  to  use  these  skills,  these  skills  and 
the  incentive  to  use  them  will  deteriorate. 

CRITICAL  RESEARCH  AND  DEVELOPMENT  ISSUES 

Following  are  major  research  and  development  issues  that  workshop  participants  thought 
must  addressed  if  Army  critical  thinking  training  was  to  be  effective. 

Development  of  valid  evaluation  methods  and  measures  of  critical  thinking 

Evaluation  is  critically  important  because  we  can’t  effectively  train  critical  thinking  if  we 
don’t  know  if  the  training  is  working.  We  need  a  more  explicit  approach  to  evaluation. 
Historically,  measurement  of  critical  thinking  has  been  problematic.  However,  without  valid 
evaluations,  we  can  t  know  if  the  training  is  effective,  what  parts  are  working,  what  needs  to  be 
changed,  and  how  good  the  instructors  are.  We  also  need  data  on  critical  thinking  measures  of 
performance  and  effectiveness  to  show  that  critical  thinking  training  is  worth  the  Army’s 

investment  of  time  and  money  and  to  convince  instructors  to  buy  into  the  critical  thinking 
concept. 

Measurement  of  critical  thinking  is  also  important  if  the  Army  culture  is  to  reward  critical 
thinking.  360  degree  assessments  do  not  have  much  to  do  with  critical  thinking.  How  do  you 
measure  critical  thinking  for  OER? 

Team  training  of  critical  thinking 

This  is  a  research  area  that  has  not  received  much  attention.  However,  since  most  of  the 
Army’s  planning  and  execution  is  done  in  the  context  of  teams,  we  need  to  know  how  individual 
critical  thinking  affects  team  performance  and  how  the  team  context  affects  individual  critical 
thinking.  How  can  critical  thinking  be  effectively  implemented  in  team  environments?  How  can 
we  train  critical  thinking  for  effective  team  performance?  What  kind  of  interpersonal  skills  are 
needed  to  implement  critical  thinking  in  teams? 
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Individual  differences  in  critical  thinking.  What  conditions  elicit  critical  thinking?  What 
conditions  stop  critical  thinking? 

An  area  of  research  that  has  received  less  attention  is  identifying  the  individual  difference 
variables  that  may  lead  to  critical  thinking.  For  example,  Fischer’s  paper  suggests  some 
predisposing  attitudes  that  may  lead  to  a  propensity  to  engage  in  critical  thinking,  such  as 
skepticism,  curiosity,  persistence  and  resistance  to  anxiety.  Additionally,  both  Fischer’s  and 
Halpem’s  papers  discuss  the  cognitive  antecedents  to  cr  ~al  thinking.  However,  the 
nomological  network  of  such  antecedents  to  critical  thin  ig  should  be  broadened  to  include 
other  important  individual  differences,  such  as  individual’s  valuation  of  critical  thinking  and 
their  motivation  to  engage  in  critical  thinking.  For  example,  given  that  the  critical  thinking  state 
is  largely  considered  to  be  unpleasant,  research  should  investigate  the  individual  differences  that 
predispose  individuals  to  engage  in  a  critical  thinking  state.  Similarly,  research  could  also  be 
directed  at  identifying  individual  differences  and  motivational  forces  that  relate  to  maintaining 
and  exiting  the  critical  thinking  state,  as  well  as  those  that  impact  the  effectiveness  of  one’s 
critical  thinking  “output.”  Only  with  more  research  into  these  areas  can  researchers  begin  to 
develop  predictors  of  individual’s  propensity  toward  critical  thinking  that  can  be  used  to  identify 
critical  thinkers  and  to  identify  those  who  would  most  benefit  from  critical  thinking  training. 

Simulations  to  explore  “what  if’  thinking  and  provide  intelligent  feedback 

We  should  invest  in  research  and  development  for  sophisticated  simulations.  Simulations 
can  create  complex  situations  and  provide  experiences  that  would  not  be  possible  otherwise.  In 
acquiring  critical  thinking  skills,  practice  and  experience  is  critical.  These  skills  can’t  be  learned 
through  lectures  or  books.  Feedback  is  also  critical  to  learning.  Distance  learning  and  self¬ 
development  software  with  critical  thinking  will  never  be  effective  without  advanced  feedback 
capabilities. 
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